Systems and methods for generating targeted media content

ABSTRACT

Systems and methods for generating targeted media content capture an image from an image capture device. The system can analyze the image to recognize an image fingerprint, such as a facial feature or a license plate. Some image fingerprints can be correlated with one another, for example a person&#39;s face can be correlated with a car license plate. The system then correlates image fingerprints with attributes, which could be used to select media content that is specialized for those attributes. The system can also collect correlative information to enhance the attribute and image fingerprint capabilities.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation-In-Part application of currentlypending U.S. patent application Ser. No. 16/142,435, filed Sep. 26,2018, and it is also a Continuation-In-Part application of currentlypending U.S. patent application Ser. No. 16/111,109, filed Aug. 23,2018, which is a Continuation-In-Part of U.S. patent application Ser.No. 16/036,625, filed Jul. 16, 2018. The subject matter of allapplications cited herein are hereby incorporated by reference in itsentirety as if set forth herein.

FEDERALLY FUNDED RESEARCH OR DEVELOPMENT

Not Applicable

BACKGROUND

The present disclosure relates generally to providing or supplementingmedia content based on one or more images captured by an image capturedevice.

There are many types of media delivery systems used by companies, suchas, for example, billboards, service kiosks, drive-through windows, andgas station pumps, that display media to consumers in a variety oflocations, such as, for example, museums, galleries, theme parks,audience centers, zoos, and roads. Often, to enhance the viewerexperience, these media delivery systems will provide media content, forexample to inform, educate or entertain a target audience, and/or toadvertise goods or services. The media content is often played back on adisplay, speakers, headphones, and/or other playback devices, such as,for example, a portable or handheld device (e.g., a tablet orsmartphone).

One aspect of providing media content to supplement a media deliverysystem is providing content that will be of interest or entertaining tothe audience members. Each audience may be made up of various membersthat have different interests and needs. Thus, those skilled in the artare constantly striving to provide systems and methods that providemedia content that supplements media delivery systems to meet the needsof each particular audience member.

SUMMARY

Systems and methods in accordance with various aspects of thisdisclosure provide targeted media content based upon a set of imagescaptured. The media content computer system is generally electronicallycoupled to any media content playback device(s) and the image-capturingdevices of the system. As used herein, an electronic or a functionalconnection is one that allows two electronic devices to transmit data toone another. An electronic connection may be wired or wireless using anysuitable data-coupling technology.

In preferred embodiments, a set of images is analyzed to determine themost effective targeted media content appropriate. One or more imagesare generally analyzed by a media content computer system to generateidentifiers as a function of image fingerprints found in a set ofimages, for example by transmitting queries to an image database thatcorrelates image fingerprints with identifiers.

The media content computer system could also be programmed to analyzeidentifiers to determine related identifiers that are not found in a setof images, for example by transmitting queries to a relationshipdatabase that correlates identifiers that are related to one another.Exemplary related identifiers include identifiers of occupants (e.g.drivers and/or passengers) that are known to ride within an identifiedvehicle, or identifiers of students that are known to follow anidentified teacher. In preferred embodiments, the media content computersystem could generate new relationships through analysis of a set ofimages, and can also create expiration times for any relationships suchthat a relationship could be deleted upon a time threshold. For example,a media content computer system could analyze a set of images thatcontain three people identified via facial recognition algorithms whoall enter a car having a license plate. Unique identifiers of each ofthe three people and the car could be associated with one another in therelationship database, such that each person has a relationship with thecar. Relationships could be set to expire as a function of an algorithm,for example a person who sits in a driver's seat could be configured tohave a permanent relationship or a relationship that expires within ayear while a person who sits in a passenger seat could have arelationship set to expire within a threshold time period (e.g. a fewdays, a week).

The media content computer system preferably uses some or all of theidentifiers to determine attributes that are related to the identifiers,for example by transmitting queries to an attribute database thatcorrelates an identifier with one or more attributes. In someembodiments, the media content computer system could further populatethe attribute database by analyzing one or more transactions. A set ofimages could be taken of an individual identified by their facialfingerprint, and recognition of an action indicative of a transactiontaking place, for example a movement to swipe an RFID (Radio FrequencyIdentifier) chip in front of a gas station pump or a movement to providea credit card to a cash register attendant. That transaction identifiercould be correlated with a transaction that is recorded by atransactional capture device, such as a cash register or a gas stationpump via any suitable means, for example via a time-stamp or a bar code,such that transaction information could then be associated with theperson's identifier in the attribute database.

With such gleaned attributes, one or more samples of media content couldbe generated, for example by transmitting queries to a media contentdatabase that correlates one or more attributes with media content. Forexample, an individual who purchases a certain type of good could beprovided media content associated with that good, or an individual whoprefers media content with subtitles could be provided media contentwith subtitles. The media content system could further monitorindividuals' actions in response to the playing media content to alterthe media content, for example an individual who is not looking at adisplay within a threshold period of time could trigger the mediacontent system to cycle the playing media content with a different mediacontent file. Attributes in the attribute database could also further bemodified as a function of observed actions.

In some embodiments, a media content computer system could be programmedto identify a transaction, for example a payment for fuel at a fillingstation or a payment for one or more items at a cash register or otherelectronic payment station. In some embodiments, the media contentcomputer system could be programmed with an advanced algorithmicanalysis program that recognizes transactional movements and goods froma detailed set of images to derive transaction attributes (e.g.identifies a facial fingerprint, identifies a purchase, and identifies agood to associate an identifier of the facial fingerprint with anattribute of that good). In preferred embodiments, the media contentcomputer system cooperates with transactional capture devices thatcapture a set of transaction attributes and a captured transactionidentifier. Such captured transaction identifiers captured by thetransactional capture device could then be matched with generatedtransaction identifiers derived from an analysis of a set of images.Contemplated transaction identifiers include time-stamp values andpartial or whole account numbers (e.g. credit card account numbers ormembership IDs). Contemplated transactional capture devices includefilling station pumps, service kiosks, and cash register computersystems. Matched transaction identifiers could then be used to update anattribute database using transaction attributes captured by atransactional capture device correlated with identifiers derived from animage fingerprint.

In one embodiment, a media content system and method may also determineplayback parameters for the media content. In accordance with someaspects, a configuration of an interactive touchscreen or other inputdevice may be modified based upon the captured image. In accordance witha number of these aspects, a subsequent image may be captured, and themedia content and/or playback parameters are updated based upon thesubsequent image. For example, an image may identify a height of aperson relative to a height of a touchscreen, which triggers the systemto move a user interface of the touchscreen to a new height as afunction of the person's height, or an image may identify a personhaving an English primary language attribute, which triggers the systemto play media non-English content with English subtitles.

A media content-providing system in accordance with this disclosurecould optionally include one or more of an exhibit control system,module, or functionality; a content control system, module, orfunctionality; an image analysis system, module, or functionality; and afacial recognition system, module, or functionality. The exhibit controlfunction may advantageously be provided by a computer system that isconnected to an image capture device (e.g., a camera) focused on anaudience area near the exhibit, and one or more media playback devices.The computer system could control the camera to capture images of theaudience area, process the image to recognize identifiers associatedwith attributes, and provide the attributes to a content control system,module, or functionality. The computer system then receives mediacontent information to download media content from a third party or itreceives media content directly. The media content is then played backby the playback devices. The media content information may includeplayback parameters for the media content, and the computer system mayadvantageously adjust the playback parameters based on information fromthe facial recognition system. The content control function may beperformed by a computer system, a database storing media contentassociated with the exhibit, and a database that stores audience memberinformation. The content control system or module receives the imagefrom the exhibit control system or module and provides the image to theimage analysis system or module and/or facial recognition system ormodule. The content control system or module then receives audienceinformation from the image analysis system or module and/or the facialrecognition system or module, and it determines the media content andplayback parameters that are sent to the exhibit control system ormodule. The image analysis system or module receives the image of theaudience area from the content control system or module, analyzes theimage, and returns audience information attributes determined based onthe image analysis to the content control system or module.

Contemplated media delivery systems further include any device suitablefor delivering media content to an audience member, preferably in avideo/audio format, although other forms of media content could be used,for example smell, taste, or tactile formats. In accordance with someaspects of this disclosure, a media delivery system includes an imagecapture device operable to obtain an image, a media content playbackdevice, a processor, and memory that stores instructions for theprocessor, media content that can be played on the playback device, anda database that stores identifier relationships between media contentand identified entities within a captured image. The instructions causeone or more processors of the media delivery system to receive the imagefrom the image capture device. The image is analyzed to determine imageidentifiers, and the image identifiers are matched with related mediacontent in the database to determine what types of media are the mostappropriate to display on the media content playback device.

In accordance with some other aspects of the disclosure, a method forproviding media content for an exhibit is performed in the followingmanner: An image of an entity is captured by an image capture device. Aprocessor performs image analysis on the captured image to identify oneor more image identifiers. The processor then matches the imageidentifiers against media content that is related with one or more ofthe image identifiers. The processor then selects media content andtransmits it to the media content playback device.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagrammatic embodiment of systems and devices that performprocesses for providing media content to supplement an exhibit inaccordance with aspects of the disclosure.

FIG. 2 is a block diagram of an exemplary computer processing system inaccordance with an aspect of the disclosure.

FIG. 3 is conceptual perspective view of an exemplary room with anexhibit including playback devices to provide supplemental media contentaccordance with an aspect of the disclosure.

FIG. 4A is a flow diagram of an overview of an exemplary process forproviding supplemental media content for an exhibit based upon an imageof an audience area of the exhibit in accordance with an aspect of thedisclosure.

FIG. 4B is a flow diagram of an exemplary process for providingsupplemental media content for an exhibit based upon visual identifiersof one or more groups identified in an image of an audience area of theexhibit in accordance with an aspect of the disclosure.

FIG. 4C is a flow diagram of an exemplary process for providingsupplemental media content for an exhibit based upon facial recognitionof audience members in an image of an audience area of the exhibit inaccordance with an aspect of the disclosure.

FIG. 5 is a block diagram of components of an exhibit control system inaccordance with an embodiment of the disclosure.

FIG. 6 is a flow diagram of an exemplary process performed by theexhibit control system to obtain and playback supplemental media contentin accordance with an aspect of the disclosure.

FIG. 7 is a flow diagram of an exemplary process performed by a contentcontrol system to provide supplemental media content to an exhibit inaccordance with an aspect of the disclosure.

FIG. 8A is conceptual diagram of a data record for an audience memberstored by an exemplary content control system for using in determiningthe proper media content to provide in accordance with an aspect of thedisclosure.

FIG. 8B is conceptual diagram of a data record for a group stored by thecontent control system of FIG. 8A for using in determining the propermedia content to provide in accordance with an aspect of the disclosure.

FIG. 9A is a flow diagram of a process performed by an exemplary contentcontrol system to obtain audience member information and generate anaudience member record in accordance with an aspect of this disclosure.

FIG. 9B is a flow diagram of a process performed by an exemplary contentcontrol system to obtain group information and generate a group recordin accordance with an aspect of this disclosure.

FIG. 10A is a flow diagram of a process performed by an embodiment ofimage analysis system to store data records of images of audiencemembers in accordance with an aspect of the disclosure.

FIG. 10B is a flow diagram of a process performed by an exemplary facialrecognition system to store data records of images of audience membersin accordance with an aspect of the disclosure.

FIG. 11 is a conceptual drawing of an exemplary facial image recordmaintained by facial recognition system in accordance with an aspect ofthe disclosure.

FIG. 12 is a conceptual diagram of exemplary modules of software forperforming facial recognition analysis on a captured image of anaudience area in accordance with an aspect of the disclosure.

FIG. 13 is a flow diagram of an exemplary process performed by a facialrecognition system to generate audience information from a capturedimage of an audience area in accordance with an aspect of thedisclosure.

FIG. 14 is a conceptual drawing of an exemplary group image analysisrecord maintained by facial recognition system in accordance with anaspect of the disclosure.

FIG. 15 is a conceptual diagram of an embodiment of functional modulesfor performing image analysis on a captured image of an audience area inaccordance with an aspect of the disclosure.

FIG. 16 is a flow diagram of an exemplary process performed by an imageanalysis module to generate audience information from a captured imageof an audience area in accordance with an aspect of the disclosure.

FIG. 17 is a flow diagram of an exemplary process performed by an imageanalysis module to analyze a captured image to identify visualidentifiers of groups in an audience area in accordance with an aspectof the disclosure.

FIG. 18 is a flow diagram of an exemplary process performed by an imageanalysis module to analyze a captured image to identify members ofgroups based upon colors or patterns of visual identifiers in accordancewith an aspect of the invention.

FIG. 19 shows a schematic of an embodiment of a media delivery system.

FIG. 20 shows an exemplary method of using the media delivery system ofFIG. 19.

FIG. 21 shows a diagrammatic representation of an embodiment of systemsand devices that perform processes for providing location informationfor groups of users in a venue in accordance with one aspect of theinvention.

FIG. 22 is a block diagram of an embodiment of a computer processingsystem in a component in accordance with an aspect of the disclosure.

FIG. 23 is conceptual perspective view of an exemplary venue havingplacement of image capture devices at various exhibits in the venue inaccordance with an aspect of the disclosure.

FIG. 24 is a flow diagram of an exemplary process for providing locationinformation about users at various locations in a venue in accordancewith an aspect of the disclosure.

FIG. 25 is a block diagram of exemplary components of an exhibit controlsystem in accordance with an aspect of the disclosure.

FIG. 26 is a flow diagram of an exemplary process performed by theexhibit control system to capture an image of registered users in anexhibit area in accordance with an aspect of this disclosure.

FIG. 27 is a flow diagram of an exemplary process performed by theexhibit control system to capture provide consumable information aboutregistered users in accordance with an aspect of this disclosure.

FIG. 28 is a flow diagram of a process performed by an exhibit controlsystem to associate an image captured at a photo opportunity are with aregistered user in accordance with an aspect of the invention.

FIG. 29 is a flow diagram of a process performed a central controlsystem to maintain and provide location information for registered usersand crowd information for locations including exhibits in accordancewith an aspect of this invention.

FIG. 30 is a flow diagram of a process performed a central controlsystem to maintain and provide identifications of consumable productsavailable to registered users in accordance with an aspect of thisinvention.

FIG. 31 is a flow diagram of a process performed by a central controlsystem to obtain and maintain captured images for photo opportunityareas associated with registered users in accordance with an embodimentof the invention.

FIG. 32 is a flow diagram of a process performed by the central controlsystem to determine whether registered users meet required criteria foran area in the venue in accordance with an aspect of the disclosure.

FIG. 33 is a flow diagram of a process performed by the central controlsystem to provide location information of registered users in a group toa group member in accordance with an aspect of the disclosure.

FIG. 34 is a conceptual diagram of a data record for a registered userstored by the central control system in accordance with an aspect of thedisclosure.

FIG. 35 is a conceptual diagram of a data record for a location in thevenue stored by the central control system in accordance with an aspectof the disclosure.

FIG. 36 is a flow diagram of a process performed by a central controlsystem to obtain user information and generate a registered user recordin accordance with an aspect of this disclosure.

FIG. 37 is a flow diagram of a process performed by a central controlsystem to provide crowd information to registered users in accordancewith an aspect of the disclosure.

FIG. 38 is a flow diagram of a process performed by a facial recognitionsystem to store data records of images of registered users in accordancewith an aspect of the disclosure.

FIG. 39 is a conceptual drawing of a facial image record maintained byfacial recognition system in accordance with an aspect of thedisclosure.

FIG. 40 is a conceptual diagram of the modules of software forperforming facial recognition analysis on a captured image of a locationin accordance with an aspect of the disclosure.

FIG. 41 is a flow diagram of a process performed by a facial recognitionsystem to generate location information from a captured image of an arealocation in accordance with an aspect of the disclosure.

DETAILED DESCRIPTION

Systems and methods in accordance with various aspects of thisdisclosure provide targeted media content based upon a set of imagescaptured. As used herein, a “set” comprises at least one item, such thata “set of images” could comprise only a single image, or several images,but never zero images. It should be appreciated that any captured,received, or generated data could comprise a single data element or aplurality of data elements.

For purposes of this discussion, “media content” comprises any type ofcontent that may be sensed by human being during playback. Examples oftypes of media content include, but are not limited to, visual, audio,tactile, vibrational, smell, taste, or any other form of media that maybe sensed by a human user. Accordingly, a media content playback devicecould be any suitable electronic device that is configured to play themedia content to a user, for example a video display, an audio speaker,a braille display, an olfactometer, or any combination thereof.

At least one display and image capturing device are preferably accessedand controlled by a media content computer system that cross-referencesinformation gleaned from a set of captured images against one or moredatabases to select targeted media content. As used herein, a “computersystem” comprises any suitable combination of computing or computedevices, such as desktops, laptops, cellular phones, blades, servers,interfaces, systems, databases, agents, peers, engines, modules, orcontrollers, operating individually or collectively. Computer systemscomprise at least a processor configured to execute softwareinstructions stored on a tangible, non-transitory computer readablestorage medium (e.g., hard drive, solid state drive, RAM, flash, ROM,etc.). The software instructions preferably configure the computersystem to execute the functionality as disclosed. In some embodiments, acomputer system may be distributed across a network, while in otherembodiments, a computer system may comprise only a single embeddedsystem.

Aspects of the disclosure may be embodied as a system, method or programcode/instructions stored on one or more machine-readable media.Accordingly, aspects may take the form of hardware, software (includingfirmware, resident software, micro-code, etc.), or a combination ofsoftware and hardware aspects that may all generally be referred toherein as a “circuit,” “module” or “system.” The functionality providedas individual modules/units in the example illustrations can beorganized differently in accordance with any one of platform (operatingsystem and/or hardware), application ecosystem, interfaces, programmerpreferences, programming language, administrator preferences, etc.

Any combination of one or more non-transitory machine-readable medium(s)may be utilized. A machine-readable storage medium may be, for example,but not limited to, a system, apparatus, or device, that employs any oneof or combination of electronic, magnetic, optical, electromagnetic,infrared, or semiconductor technology to store program code. Morespecific examples (a non-exhaustive list) of the machine-readablestorage medium would include the following: a portable computerdiskette, a hard disk, a random-access memory (RAM), a read-only memory(ROM), an erasable programmable read-only memory (EPROM or Flashmemory), a portable compact disc read-only memory (CD-ROM), an opticalstorage device, a magnetic storage device, or any suitable combinationof the foregoing. In the context of this document, a non-transitorymachine-readable storage medium may be any tangible medium that cancontain, or store a program for use by or in connection with aninstruction execution system, apparatus, or device. A non-transitorymachine-readable storage medium can be machine-readable, but preferablyprogram instructions are not stored on a non-transitory storage medium.

A machine-readable signal medium may include a propagated data signalwith machine-readable program code embodied therein, for example, inbaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including, but not limited to,electro-magnetic, optical, or any suitable combination thereof. Amachine-readable signal medium may be any machine-readable medium thatis not a machine-readable storage medium and that can communicate,propagate, or transport a program for use by or in connection with aninstruction execution system, apparatus, or device.

Program code such as the “C” programming language, assembly language, orsimilar programming languages embodied on a machine-readable medium maybe transmitted using any appropriate medium, including but not limitedto wireless, wireline, optical fiber cable, RF, etc., or any suitablecombination of the foregoing.

Computer program code for carrying out operations for aspects of thedisclosure may be written in any combination of one or more programminglanguages, including an object oriented programming language such as theJava® programming language, C++ or the like; a dynamic programminglanguage such as Python; a scripting language such as Perl programminglanguage or PowerShell script language; and conventional proceduralprogramming languages, such as the “C” programming language or similarprogramming languages. The program code may execute entirely on astand-alone machine, may execute in a distributed manner across multiplemachines, and may execute on one machine while providing results and oraccepting input on another machine.

The program code/instructions may also be stored in a non-transitorymachine-readable medium that can direct a machine to function in aparticular manner, such that the instructions stored in themachine-readable medium produce an article of manufacture includinginstructions which implement the function/act specified in the flowchartand/or block diagram block or blocks.

As used herein, an “image fingerprint” is any portion of an image thatcan be analyzed to provide a characteristic. The characteristics couldbe unique, for example a facial fingerprint used to perform facialrecognition on a photo of a person's face, or they could be non-unique,for example a uniform worn by one of a plurality of people within agroup. Likewise, identifiers could also be unique, for example aperson's social security number or a UID (Unique Identifier assigned bythe system), or non-unique, for example a name of a school a group ofstudents attend. Contemplated image fingerprints include facialfingerprints, license plate numbers, logos, bar codes,clothing/accessories, or any other suitable image that can be taggedwith a feature. Contemplated identifiers include social securitynumbers, UIDs, license plate alphanumeric codes, car make/model, andgroup names (e.g. name of school, name of class, name of company).

As used herein, an “attribute” comprises any characteristic, and couldbe saved in any suitable form, for example a numerical value or analphanumeric string. Contemplated attributes include preferences forgoods and/or services and demographics.

Flowcharts are provided to aid in understanding the illustrations andare not to be used to limit scope of the claims. The flowcharts depictexemplary operations that can vary within the scope of the claims.Additional operations may be performed; fewer operations may beperformed; the operations may be performed in parallel; and theoperations may be performed in a different order. Each block of theflowchart illustrations and/or block diagrams, and combinations ofblocks in the flowchart illustrations and/or block diagrams, can beimplemented programmatically by executing program code on a computersystem using one or more processors.

Plural instances may be provided for components, operations orstructures described herein as a single instance. Boundaries betweenvarious components, operations and data stores may be definedarbitrarily. Particular operations are preferably illustrated in thecontext of specific illustrative configurations. Other allocations offunctionality are envisioned and may fall within the scope of thedisclosure. In general, structures and functionality shown as separatecomponents in the example configurations may be implemented as acombined structure or component. Similarly, structures and functionalityshown as a single component may be implemented as separate components.These and other variations, modifications, additions, and improvementsmay fall within the scope of the disclosure.

Cloud Computing: The system and techniques described above areapplicable and useful in the cloud computing environment. Cloudcomputing means computing capability that provides an abstractionbetween the computing resource and its underlying technical architecture(e.g., servers, storage, networks), enabling convenient, on-demandnetwork access to a shared pool of configurable computing resources thatcan be rapidly provisioned and released with minimal management effortor service provider interaction. The term “cloud” is intended to referto a plurality of disparate computer systems electronically coupledusing a network, and cloud computing allows shared resources, forexample, software, data, processors, bandwidth, and information to beavailable, on-demand, like a public utility.

Exemplary cloud computing providers deliver business applications onlinewhich are accessed from another web service or software like a webbrowser, while the software and data are stored remotely on servers. Thecloud computing architecture can use a layered approach for providingapplication services. A first layer is an application layer that isexecuted at client computers. After the application layer, could be acloud platform and cloud infrastructure, followed by a “server” layerthat includes hardware and computer software designed for cloud specificservices. The various disclosed systems can be a part of the serverlayer for using the innovative techniques described above.

FIG. 1 illustrates a system 100 for providing media content tosupplement an exhibit in accordance with an aspect of the disclosure.The system 100 includes a facial recognition module 102 and/or an imageanalysis module 106; a content control module 104; and exhibit controlmodule 108, all of which are communicatively connected to one another bya network 110. A portable personal communication device 120 and acomputer 125 may also be connected to the network 110. Although shown asseparate devices or functionalities in FIG. 1, the facial recognitionmodule 102, one or more of the content control module 104, the imageanalysis module 106, and the exhibit control module 108 may be providedby a single computing system. Alternatively, the processes that provideone or more of the facial recognition module 102, the content controlmodule 104, the image analysis module 106, and the exhibit controlmodule 108 may be distributed across multiple systems that arecommunicatively connected via the network 110.

The facial recognition module 102 may be implemented or functionalizedby a computer system that includes a memory and a processing unit toperform the processes for providing facial recognition, for example bycapturing a set of images and cross-referencing facial fingerprintsagainst a database that contains relationships between facialfingerprints and identifiers. The computer system that implements thefacial recognition module, functionality, or system may include one ormore servers, routers, computer systems, and/or memory systems that arecommunicatively connected via a network to provide facial recognitionand/or other image analysis, and to return identifiers that can be usedto derive appropriate media content.

The content control module 104 may be implemented or functionalized by acomputer system that includes a memory and a processing unit to performprocesses for storing and providing media content for one or moreexhibits in a venue. The content control module 104 may alsoadvantageously store and update audience information for use indetermining the media content to provide to an exhibit. The contentcontrol functionality may be provided by a central control system forthe venue. Specifically, the content control module 104 may beimplemented or functionalized by a system that includes one or moreservers, routers, computer systems, and/or memory systems that arecommunicatively connected via a network to store and provide mediacontent for one or more exhibits in the venue, as well as to store andupdate audience information for use in determining the content toprovide to an exhibit.

The image analysis module 106 may be implemented or functionalized by acomputer system that includes a memory and a processing unit to performthe processes for providing image analysis. The computer system thatimplements the image analysis module, functionality, or system mayinclude one or more servers, routers, computer systems, and/or memorysystems that are communicatively connected via a network to providefacial recognition and/or other image analysis.

The exhibit control module 108 may be implemented or functionalized by acomputer system that controls devices in the exhibit area that includean image capture device and various playback devices for media contentthat supplements the exhibit. Advantageously, one computer system maycontrol devices for more than one exhibit. In specific embodiments, theexhibit control module 108 may be implemented or functionalized by asystem that includes one or more servers, routers, computer systems,memory systems, an image capture device, and/or media playback devicesthat are communicatively connected via a local network to obtain andpresent media content for the exhibit.

The network 110 may advantageously be the Internet. Alternatively, thenetwork 110 may be a Wide Area Network (WAN), a Local Area Network(LAN), or any combination of Internet, WAN, and LAN that can be usedcommunicatively to connect the various devices and/or modules shown inFIG. 1.

The portable personal communication device 120 may a smart phone,tablet, Personal Digital Assistant (PDA), a laptop computer, or anyother device that is connectable to the network 110 via wirelessconnection 122. The computer 125 may advantageously connect to thenetwork 110 via either a conventional “wired” or a wireless connection.The computer 125 may be, for example, a desktop computer, a laptop, asmart television, and/or any other device that connects to the network110. The portable personal communication device 120 and/or the computer125 allow a user to interact with one or more of the above-describedmodules to provide information such as, for example, personalinformation to be added to audience member information of the user. Insome embodiments, the portable personal communication device 120 or amedia delivery system 128 may be used as the playback device of thesupplemental media content for an exhibit.

Although a particular system of devices and/or functional modules isdescribed above with respect to FIG. 1, other system architectures that,add, remove, and/or combine various devices and/or modules may be usedto perform various processes in accordance with various other aspects ofthe disclosure.

FIG. 2 is a high-level block diagram showing an example of thearchitecture of a processing system 200 that may be used according tosome aspects of the disclosure. The processing system 200 can representa computer system that provides a facial recognition functionality, acontent control functionality, an image analysis functionality, anexhibit control functionality, and/or other components orfunctionalities.

Processing system 200 includes one or more processors 205 in operativecommunication with memory 210 and coupled to a bus system 212. The bussystem 212, as shown in FIG. 2, is a schematic representation of any oneor more separate physical buses and/or point-to-point connections,connected by appropriate bridges, adapters and/or controllers. The bussystem 212, therefore, may include, for example, a system bus, aPeripheral Component Interconnect (PCI) bus, a HyperTransport orindustry standard architecture (ISA) bus, a small computer systeminterface (SCSI) bus, a universal serial bus (USB), or an Institute ofElectrical and Electronics Engineers (IEEE) standard 1394 bus (sometimesreferred to as “Firewire”).

The one or more processors 205 are the central processing units (CPUs)of the processing system 200 and, thus, control its overall operation.In certain aspects, the one or more processors 205 accomplish this byexecuting software stored in memory 210. The processor(s) 205 may be, ormay include, one or more programmable general-purpose or special-purposemicroprocessors, digital signal processors (DSPs), programmablecontrollers, application specific integrated circuits (ASICs),programmable logic devices (PLDs), or the like, or a combination of suchdevices.

Memory 210 represents any form of random access memory (RAM), read-onlymemory (ROM), flash memory, or the like, or a combination of suchdevices. Memory 210 includes the main memory of the processing system200. Instructions 215 implementing the process steps of described belowmay reside in memory 210 and are executed by the processor(s) 205 frommemory 210.

Also advantageously connected operatively to the processor(s) 205through the bus system 212 are one or more internal or external massstorage devices 220, and a network interface 222. The mass storagedevice(s) 220 may be, or may include, any conventional medium forstoring large volumes of data in a non-volatile manner, such as one ormore solid state, magnetic, or optical based disks. The networkinterface 222 provides the processing system 200 with the ability tocommunicate with remote devices (e.g., storage servers) over a network,and may be, for example, an Ethernet adapter, a Fiber Channel adapter,or the like.

The processing system 200 also advantageously includes one or moreinput/output (I/O) devices 217 operatively coupled to the bus system212. The I/O devices 217 may include, for example, a display device, akeyboard, a mouse, etc.

FIG. 3 illustrates an exhibit display area in accordance with an aspectof the invention. As shown in FIG. 3, an exhibit 315 is located in aroom. For example, the exhibit 315 may be mounted on a wall of a room(as shown), placed on the floor, or hanging from the ceiling.Alternatively, the exhibit 315 may be a stage or other raised platformwhere performances by actors, artists, musicians, or others may bestaged.

To provide supplemental media content, one or more media playbackdevices may be provided to present the supplemental media content to anaudience. For example, a personal device, such as a smart phone, tablet,or other media playback device may be carried or worn by one or moreaudience members and/or exhibit staff members. The personal devices maycommunicate with the exhibit control module via a wireless connection,either directly to the exhibit control module, or through a networkconnection in accordance with various aspects to obtain and/or presentthe supplemental media content.

In FIG. 3, the playback devices are shown as a display 305 and speakers320. The display 305 may be a monitor or other video playback devicethat is located proximate the exhibit 315 to display video content ofthe supplemental media content for the exhibit 315. Speakers 320 areauditory playback devices that may advantageously be mounted to thewall, or standing proximate the wall, under the display 305, orelsewhere in the room, and that play back auditory content in thesupplemental media content. In general, the display 305, speakers 320,and/or other playback devices may be located or mounted anywhereproximate the exhibit 315, and they are advantageously placed to providesufficient coverage of an audience area 325 to allow the desired numberof audience members to view, hear, and/or in some other way sense thepresentation of the media content.

An audience area 325 is defined proximate the exhibit 315. In FIG. 3,the audience area 325 is the floor in front of the exhibit 315; however,the audience area 325 may be any defined area where an audience may beexpected to stand, sit, or otherwise view the exhibit. For example, theaudience area 325 may be benches or seats in front of the exhibit 315.In some embodiments, a sensor 330, such as, for example, a pressuresensor, a motion detector, or any other type of sensor that senses thepresence of at least one audience member, is located in or near toaudience area 325.

An image capture device 310, such as, for example, a camera (preferablya video camera), is located proximate the exhibit 315, e.g., in thewall, and it is focused on audience area 325. The image capture device310 captures still images and/or video images of the audience as theaudience views the display 305 and/or the exhibit 315. Although shown aswall-mounted proximate the exhibit 315, the image capture device 310 maybe placed anywhere in the area of the exhibit 315 that will allow thedevice to capture images of at least a portion, if not all, of theaudience members that are in and/or proximate to the audience area 325.

Although an exemplary exhibit area in accordance with an aspect of theinvention is described above with reference to FIG. 3, otherconfigurations that add, remove, combine, and/or move componentsrelative to one another are possible.

FIG. 4A illustrates a flow diagram of a general method for providingsupplemental media content for an exhibit using image processing of acaptured image of the audience area of the exhibit in accordance withanother aspect of the invention. Process 4000 receives information aboutan audience member (4005). In accordance with some embodiments, theinformation may be particular to the individual audience member. Inaccordance with some other embodiments, the information may pertain togroup that includes the audience member. For purposes of thisdiscussion, a group may be any set of audience members that have commoncharacteristics that may be used to determine media content to provide.Examples of groups may include, but are not limited to, classes, tourgroups, families, and people with similar disabilities.

A visual identifier is then assigned for the audience members (4010). Inaccordance with some embodiments, a visual identifier may be particularto an individual audience member. For example, a facial image of theaudience member may be assigned as a visual identifier of the audiencemember in many of these embodiments. In accordance with some otherembodiments, the visual identifier may be some sort of visual identifierthat is assigned to each member of a related group of audience members.For example, the visual identifier may be, but is not limited to, aparticular color or pattern for garments worn by the group or may be acolor, pattern, or symbol on a lanyard, badge, or tag worn or held byeach member of the group that is distributed to each member of the groupby the venue.

A record that includes an identifier associated with the audiencemember, a visual identifier, and information relevant in determining themedia content to present is stored (4015). In accordance with someembodiments, the record is a group record that identifies a group nameor other group identifier, the visual identifier associated with thegroup, and group information that is information relevant in determiningthe media content. In accordance with some other embodiments, the recordmay be an audience member record that stores a name or other identifierof an individual audience member, a facial image or some otherparticular visual identifier of the audience member, and memberinformation relevant in determining the media content for the audiencemember.

The process 4000 captures an image of audience members in an audiencearea proximate the exhibit (4020). An image analysis process is thenperformed on the captured image to identify each visual identifier inthe captured image (4025). As discussed in below, two examples of imageprocessing that may be used include color, pattern, or symbolrecognition for group visual identifiers, and facial recognition forfacial images of audience members. The computer system can be programmedto derive such information from a set of images, and make decisions uponcross-referencing that information against a table or a database. Forexample, the computer system could analyze a set of images to determinewhat color a majority of entities wear, what pattern a majority ofentities have on a tag, what symbol from a set of recognized symbolsexist in a set of images, and/or could be programmed to derive facialfingerprints from a set of images. The member and/or group informationfor each identified visual identifier (e.g., with appropriate userinformation) is obtained (4030). Demographic information and,optionally, other audience-related information for the audience as wholemay also be determined or obtained by the image analysis device ormodule, or the facial recognition device or module (4035). Identifierinformation could be entered into a table or a database that matchesidentifiers against media content (e.g. show media content A to groupswhere a majority of entities wear red, show media content B to groupswhere a majority of entities wear blue), or into a table or databasethat matches identifiers against attributes, which could then be used asa query in another database to select media content (e.g. square symbolsare worn by English-speakers and triangles are worn by Spanish speakers.Media content in English should be shown to English speakers while mediacontent in Spanish should be shown to Spanish speakers). The mediacontent to present to the audience is then determined based on theobtained group and/or user information and/or from the determineddemographic information for the audience (4040). In accordance with someaspects, playback parameters for each piece of media content to beprovided may also be selected as a function of attributes that arederived either directly (e.g. direct analysis of a set of images) orindirectly (e.g. indirect analysis by cross-referencing information fromthe image against one or more databases). The media content and/orplayback parameters can then be provided to the exhibit control deviceor module for playback using the media playback devices (4045), afterwhich the process 4000 ends.

FIG. 4B illustrates a flow diagram of a process 400 for providingsupplemental media content for an exhibit in accordance with an aspectof the invention based upon group information and visual identifier thatidentify an audience member as part of a particular group. In someparticular embodiments, the image processing performed is colorrecognition. The process 400 captures an image of audience members in anaudience area proximate the exhibit (405).

Image analysis is then performed on the captured image of the audiencearea (410) to identify (e.g., with appropriate group information)current audience information (415). Demographic information and,optionally, other audience-related information for the audience as awhole may also be determined or obtained by the above-mentioned imageanalysis device or module 106 (420). The media content to present to theaudience is then determined based on the current audience informationidentified from the captured image of the audience area and/or from thedetermined demographic information for the audience (425). In accordancewith some aspects, playback parameters for each piece of media contentto be provided may also be determined or obtained. The media contentand/or playback parameters are provided to the exhibit control device ormodule for playback using the media playback devices (430), after whichthe process 400 ends.

FIG. 4C illustrates a flow diagram of a process for providingsupplemental media content for an exhibit using facial recognition ofaudience members in the captured image of the audience area inaccordance with another aspect of the invention. The process 450captures an image(s) of audience members in an audience area proximatethe exhibit (455). The captured image(s) or facial fingerprints derivedfrom the image(s) may advantageously be provided to a facial recognitiondevice or module (460). The facial recognition device or moduleidentifies the desired portions of the captured image of the audiencearea that include the facial image of one or more audience members(465).

In embodiments using a facial recognition process, the facialrecognition function could be performed on each identified portion ofthe captured image to identify (e.g., with appropriate user information)each audience member (470). Demographic information and, optionally,other audience-related information for the audience as whole may also bedetermined or obtained by the facial recognition device or module (475),preferably by cross-referencing identified information against an image,relationship, and/or attribute database. The media content to present tothe audience is then determined based on information gleaned from a setof images, for example an identifier of a person (cross-referencedagainst a table or database that associates that person's identifierwith the media content), an attribute of a person (cross-referencedagainst a table or database that associates that person's identifierwith an attribute), or a relationship of a person (cross-referencedagainst a table or database that associates that person's identifierwith another identifier) the audience members identified from theportions of the images that include a face and/or from the determineddemographic information for the audience (480). In embodiments that useboth a person's identifier and demographic information, both of thosecould be used as queries in an attribute database that returns mediacontent or attributes that could be aggregated or weighted in anysuitable manner. In accordance with some aspects, playback parametersfor each piece of media content to be provided may also be determined.The media content and/or playback parameters are provided to the exhibitcontrol device or module for playback using the media playback devices(485), after which the process 450 ends.

The above describes an overall process for providing media content tosupplement an exhibit in accordance with one aspect of the disclosure.However, other processes that add, combine, remove, and/or reorder thesteps of the process are possible.

As discussed above, in embodiments employing a facial recognitionfunction, as well as in embodiments employing recognition of a color,pattern, or symbol, an exhibit control device or module captures theimages of the audience and plays back the media content that is selectedbased upon the captured image. FIG. 5 is a block diagram of thecomponents of an exhibit control device or module 500 which, inaccordance with an aspect of the disclosure, includes a controller 505,an image capture device 510, a display 515, and an audio system 520.

The controller 505 may be implemented as a processing system thatcontrols the image capture device 510 in capturing images of theaudience area to obtain the media content information provided basedupon analysis of the captured image. In accordance with some aspects,the controller 505 may also control one or more components of theexhibit. These components may include, for example, valves, hydrauliclifts, animatronics that provide motion in the exhibit, and any othercomponents that receive instructions to perform a task to facilitate thepresentation of the exhibit. In some other aspects, the control systemfor more than one exhibit may be provided by a processing system.

The image capture device 510 may be a camera that captures still imagesand/or a video camera that captures video images. In the exemplaryembodiment shown in FIG. 5, the image capture device 510 is a separatedevice including a processing system that is communicatively connectedto the controller 505 via a wireless or wired connection. In some otheraspects, the image capture device 510 is an I/O device of the processingsystem or module including the controller 505. As discussed above, theimage capture device 510 is positioned such that the device is focusedon the audience area in a manner to capture images that include facialimages of the audience, and/or images of a specific color, pattern, orsymbol associated with members of the audience. The image capture device510 may also capture, record, or otherwise provide other information,such as depth information for the imaged objects.

The display 515 is communicatively connected to the controller 505. Thedisplay 515 may, in some embodiments, be a monitor that is controlled bythe processing system of the controller 505. In accordance with someother aspects, the display 515 may be one or more signs that are lightedby a lighting element that is controlled by the controller 505.Alternatively, the display 515 may be a touch screen that allowsinteraction with an audience member.

The audio system 520 may include one or more speakers that are placedaround the exhibit and/or audience area, and it may further include aprocessing system communicatively connected to the controller 505. Insome embodiments, the audio system may include an audio transducerconfigured as an I/O device of the controller 505.

Although an exemplary embodiment of an exhibit control device or moduleis described above with respect to FIG. 5, other embodiments that add,combine, rearrange, and/or remove components are possible.

FIG. 6 illustrates a flow diagram of a process 600 performed by theexhibit control device or module to provide supplemental media contentin accordance with an aspect of this disclosure. In the process 600, anaudience is detected in the audience area (605) by, for example, motionsensors, heat sensors, and/or any other type of sensor that may detectthe presence of one or more audience members in the audience area.

An image is captured of the audience area (610), for example, inresponse to the detection of one or more audience members in theaudience area. Alternatively, the image capture device may periodicallycapture an image at pre-defined intervals of time, or a video feed ofthe audience area may be continuously captured.

The captured image is transmitted to a content control device or module(615), optionally with other information about the image. Such otherimage information may include, for example, camera settings, depthinformation, lighting information, and/or other like information relatedto the image. The image information may be transmitted separately, or itmay be transmitted in or with the captured image. Optionally, a videofeed may be provided to the content control device or module. Theexhibit control device or module may optionally monitor a video feed andonly send an image that includes audience members that is taken from thefeed when an audience is detected in the audience area. The exhibitcontrol device or module may optionally perform image processing toimprove image quality prior to transmitting the image, and/or it mayoptionally isolate facial images from the captured image and send onlyportions of the image that include facial images to the content controldevice or module.

The exhibit control device or module receives media content information(620) to supplement the exhibit that is determined based upon thecaptured image, as discussed further below. The media contentinformation advantageously includes the media content to present, and itmay also include identifiers, such as, for example, internet addresses,file directory identifiers, or other identifiers that may be used toobtain the media content and/or stream the content from an identifiedcontent provider. The video content information may optionally includeplayback parameters for adjusting the parameters of the playback devicesto provide the desired playback. For example, the media contentinformation, may include brightness, contrast, resolution or otherinformation for video playback, and/or it may include volume and/orbalance information for an audio playback.

The media content is then obtained (625), e.g., by being read frommemory in the exhibit control device or module, and/or by being receivedfrom one or more specific media content storage systems. The mediacontent may optionally be streamed using adaptive bit rate streaming orsome other streaming technique from a content provider.

The playback parameters of the individual playback devices may then beadjusted based on the received media content information (630), and themedia content is then presented by the playback devices (635), at whichpoint the process 600 may end. However, in some embodiments, the processmay be periodically repeated during playback to update the media contentbeing presented to account for the composition of the audience changingas audience members arrive and depart during the playback.

Although an exemplary process performed by the exhibit control device ormodule to provide media content to supplement an exhibit in accordancewith aspects of this disclosure is discussed above with respect to FIG.6, other processes performed by the exhibit control device or module toprovide the media content that add to, combine, rearrange, or remove anyof the described steps are possible and are considered within the scopeof this disclosure.

FIG. 7 illustrates a flow diagram of a process 700 performed by thecontent control device or module to determine the media content toprovide to the exhibit control device or module, based upon the capturedimage. The process 700 may be performed for each image received. Thus,the process 700 may be performed once to determine the media content topresent at one time in accordance, or, alternatively, the process 700may be periodically performed during the presentation of media contentto update the media content being presented to account for changes inthe audience of the exhibit over time.

In the process 700, a captured image of the audience area is receivedfrom an exhibit control device or module (705). As discussed above,additional image information may optionally be received with the image.In some embodiments, the image may then be provided to a facialrecognition device or module and/or an image analysis device or modulefor image analysis (710). The content control device or module may dosome image processing prior to providing the image to the facialrecognition device or module/and or image analysis device or module. Theanalysis may include, for example, isolating a facial image and/or avisual group identifier (e.g. a pre-defined color, pattern, or symbol)in the image, modifying the image to improve image quality, and/oranalyzing the image to determine or obtain other image information. Insome embodiments, such other image information may be provided by thecaptured image to the facial recognition device or module and/or theimage analysis module.

The process 700 receives audience information that may includeidentifiers of audience members and/or groups identified in the capturedimage (715). The identifiers may be from audience information that thecontent control device or module, or some other system, device ormodule, has previously provided to the image analysis device or moduleand/or the facial recognition device and/or module, as discussed furtherbelow. In some aspects, the identifiers may be provided in a list ofaudience members and/or groups identified. Demographic information forthe audience may also be received (720). The demographic information isinformation about the characteristics of the audience that the imageanalysis device or module and/or facial recognition device or modulegenerates during analysis of the image. The demographic information maybe in the form of a list for each audience member, or it may be in theform of a number representing a quantification of one or more particularcharacteristics. The demographic information may include, for example,the ages, nationalities, races, heights, and/or genders of the people inthe audience. Other audience information may optionally be provided,such as the general emotional state of the audience even of individualaudience members.

The content provider device or module then obtains the audience memberinformation and/or group information associated with each identifierreceived (725). The audience member information may be information aboutthe identified audience member stored by the content provider device ormodule that provides insight into the interests and requirements of theparticular audience member, thereby indicating the media content thatwill be of likely interest to the member. The group information may beinformation about the identified group stored by the content providerdevice or module that provides insight into the interests andrequirements of the particular audience member, thereby indicating themedia content that will be of interest to the member.

FIG. 8A illustrates an example of an audience member record maintainedby the content provider device or module in accordance with an aspect ofthis disclosure. The audience member record 800 advantageously includesan identifier 805, such as a name or member number for the audiencemember. The record 800 may also include a facial image 810 of the memberthat the audience member either has provided to the content providerdevice or module, or that was captured from the audience member during aregistration process. The record 800 also includes fields for particularinformation about the audience member that may be used to determinemedia content that may be of the most interest to the audience member.The fields in the record 800 may advantageously include fields for oneor more personal characteristics, such as, for example, the member's age815, the member's education level 820, the member's height 825, themember's particular interests 830, any special needs of the member 835,and the primary language used by the member 840. Examples of particularinterests may include, for example, areas of study (such as science andhistory) that the member is interested in understanding. Examples ofspecial needs may include, for example, any visual aids, audio aids,and/or other aids that the user may need to perceive the media content,and requirements, such as specially-accessible inputs that a member mayneed to interact with the media content owing to a physical limitation.Each record may optionally include other fields and/or subfields thatdefine particular categories in these fields that may be used todetermine the proper media content to provide, and/or presentationrequirements that may be needed by the playback device for the member tobest experience the content.

FIG. 8B illustrates an example of a group record 850 maintained by thecontent provider device or module in accordance with another aspect ofthis disclosure. The group record 850 advantageously includes a groupidentifier 855, such as a name or group number for the group. The record850 also includes a group visual identifier 860 of the member that theaudience member either has provided to the content provider device ormodule, or that was captured from the audience member during aregistration process. The record 850 also includes fields for particularinformation about the group that may be used to determine media contentthat may be of the most interest to the group members. The fields in therecord 850 may advantageously include fields for one or more groupcharacteristics, such as the group's age level or average age 865, thegroup's education level 870, the group's particular interests 875, anyspecial needs of the group members 880, and the primary language used bythe group members 885. Each record may optionally include other fieldsand/or subfields that define particular categories in these fields thatmay be used to determine the proper media content to provide and/orpresentation requirements that may be needed by the playback device forthe member to best experience the content.

Returning to the process 700 shown in FIG. 7, the process 700 uses theaudience member information of each identified audience member and/oridentified group; and/or the demographic information to determine themedia content to present to supplement the exhibit (730). In situationsin which the media content is to be played back by the personal devicesof audience members, the process 700 may optionally use only the memberinformation of a particular member to determine the media content toprovide to that member. In some embodiments, the demographic informationwill be used to determine the content to provide even if there is nospecific audience member record for the identified audience member.

In accordance with some aspects, the group, audience member, and/ordemographic information may be applied to an algorithm that thendetermines the media content that will be of most interest to thebroadest range of audience members. The algorithm, for example, may bean artificial intelligence algorithm, such as, for example, a neuralnetwork algorithm that takes at least a portion of the audience memberand/or demographic information available and selects the media contentavailable for the exhibit that will appeal to the greatest number ofaudience members. For example, the algorithm may choose an audiopresentation in a language that is used by the greatest number ofidentified audience members, or a language determined by the greatestnumber of a particular nationality identified in the demographic orgroup information. The algorithm may then select a closed caption trackfor the language used by the second greatest number of audience membersor another group.

The subjects covered by the media content provided may be determined toappeal to the greatest number of audience members in accordance withsome aspects. For example, the algorithm may determine that most of theaudience is comprised of members interested in the scientific aspect ofthe exhibit as opposed to the historical aspect. As such, the algorithmselects video and audio media content directed to the scientific aspectsof the exhibit. The algorithm may also consider the age of the audiencemembers in selecting the content. For example, the algorithm may selectcontent directed to younger students if the average age of the audienceis younger, and more mature content if the audience average age isdetermined to be in the adult range.

Furthermore, the algorithm may weight some of the audience memberinformation based upon quality of service parameters. For example, someaudience members may have bought a subscription to a service thatentitles them to have preferential treatment over other audiencemembers. As such, the information for these members may be given addedweight in the algorithm when determining the content to provide.

In accordance with some aspects, the algorithm may give more or lessweight to the information of the identified members than to thedemographic information of the entire audience. Alternatively, thealgorithm may give more weight to the demographic information to try toappeal to the greatest number of audience members.

In accordance with some aspects, the special needs of an audience membermay include a time allocation to spend at a particular exhibit or at thevenue as a whole. As such, the algorithm may use this time allocationinformation to select media content that has a playback time thatconforms to the time allocation requirements of one or more audiencemembers. In some of these aspects, the media content may also includesuggestions guiding the audience member(s) to other exhibits in order toguide the member through the venue in the allocated time and/or see theexhibits that most interest the member(s).

Once the algorithm has determined the media content to provide and/orthe playback parameters that meet the needs of the audience, the mediacontent information and/or playback information is generated andprovided to the exhibit control device or module (735), at which pointthe process 700 may end. As discussed above, the process 700 may beperiodically repeated to update the media information and/or playbackparameters to account for the changing composition of the audience.

An exemplary process for selecting the media content to supplement anexhibit performed by a content control device or module in accordancewith an embodiment of the disclosure is described above with referenceto FIG. 7. However, other processes for selecting the media content thatadd, combine, rearrange, and/or remove one or more steps described aboveare possible in accordance with other embodiments.

In some embodiments, the analysis of the captured image of the audiencearea may be performed by a facial recognition system (i.e., device ormodule) in accordance with various aspects of the disclosure. In orderto perform facial recognition, the facial recognition device or moduleneeds facial images of the audience members to perform comparisons. Inaccordance with some aspects of the disclosure, the facial image of amember is provided by the audience member and/or captured by the systemduring a registration process used to generate an audience memberrecord, such as the record described above with respect to FIG. 8A. Inaccordance with some other embodiments, a group visual identifier suchas, for example, one or more of a color, a symbol, or a pattern isprovided by a user generating the group record or is generated elsewhereand is to generate a group record during a registration process. Thecolor, pattern, or symbol may be provided to each individual in aparticular audience in the form of an article of clothing (e.g., a hat,T-shirt, or scarf), a badge, a pin, a lanyard, a flag, a banner, aballoon, or any other appropriate item that may be worn or carried bythe individuals in the audience. The registration process may beperformed by a central control system or the content control device ormodule in accordance with various aspects of this disclosure. The facialimage and/or an identifier of the audience member, and/or a visual groupidentifier, is then provided by the registration process to the facialrecognition device or module and/or an image analysis module.

FIG. 9A illustrates a flow diagram of a registration process 900performed by a central control system or the content control device ormodule in accordance with an aspect of this disclosure in which facialrecognition of audience members is used. In the registration process900, a facial image of the audience member that is registering with thesystem is received (905). For example, the audience member may provide afacial image stored on the user device that audience member is using toregister. In that situation, the process 900 may issue a command (forexample, by a wireless communication) that directs the user device tocapture the facial image using an image capture device associated withthe user device, and to provide the image to the process 900. Theprocess 900 may also receive audience member information for the member(910). In accordance with some aspects, the registering member may inputthe information to a personal device that provides the information tothe process 900. The audience member information may include at least aportion of the information discussed above with reference to FIG. 8A.However, the information may also include any information that may beneeded to select media content using a particular algorithm.

An audience member record that includes the received audience memberinformation and the captured facial image is generated (915) and storedin an audience member database (920). The captured facial image and anidentifier of the audience member is provided to the facial recognitiondevice or model (925), and the process 900 may then end.

An exemplary process for registering an audience member in accordancewith embodiments of the disclosure is described above with respect toFIG. 9A. Other registration processes that add, combine, rearrange,and/or remove one or more steps described above are possible inaccordance with other embodiments.

FIG. 9B illustrates a flow diagram of a registration process 950performed by a central control system or the content control device ormodule in accordance with an aspect of this disclosure using groupidentifiers of groups of audience members to determine audienceinformation. In the registration process 950, visual identifierinformation of the group that is registering with the system is received(955). For example, the members of the group may be provided, forregistration, with an item to be carried or worn and that displays aparticular color, symbol, or pattern, as described above. The process950 may also receive group information for the members (960). Inaccordance with some aspects, a user registering the group may input theinformation to a personal device that provides the information to theprocess 900. The group information may include at least a portion of theinformation discussed above with reference to FIG. 8B. However, theinformation may also include any information that may be needed toselect media content using a particular algorithm.

A group record that includes the received group identifier informationand the group visual identifier for the group is generated (965) andstored in a group database (970). The group visual identifierinformation and an identifier of the group is provided to the imageanalysis device or module (975), and the process 900 may then end.

An exemplary process for registering a group in accordance withembodiments of the disclosure is described above with respect to FIG.9B. Other registration processes that add, combine, rearrange, and/orremove one or more steps described above are possible in accordance withother embodiments.

FIG. 10A illustrates a process 1000 performed by the facial recognitiondevice or module in response to receiving a facial image and identifierof an audience member in accordance with some embodiments of thisdisclosure that use facial recognition to determine audienceinformation. In the process 1000, the facial recognition device ormodule receives a facial image and identifier of an audience member froma central control system or content control device or module (1005). Afacial recognition record is generated and stored (1010). The generationof the facial recognition record may include, for example, analyzing theimage to generate facial parameters that may be used for imagecomparisons during the facial recognition process, as discussed furtherbelow.

An exemplary process for obtaining audience member facial images in afacial recognition system in accordance with embodiments of thedisclosure is described above with respect to FIG. 10A. Other processesfor obtaining facial images that add, combine, rearrange, and/or omitone or more steps described above are possible in accordance with otherembodiments.

FIG. 10B illustrates a process 1050 performed by the image analysisdevice or module in response to receiving a group identifier and groupvisual identifier information of a group in accordance with someembodiments of this disclosure that use image analysis to determineaudience information. In the process 1050, the image analysis device ormodule receives a group identifier and group visual identifierinformation of a group from a central control system or content controldevice or module (1055). A group record is generated and stored (1060).The generation of the group record may include, for example, analyzingan image of the group visual identifier to generate image parametersthat may be used for image comparisons during the image analysisprocess, as discussed further below.

An exemplary process for obtaining group information in an imageanalysis system in accordance with embodiments of the disclosure isdescribed above with respect to FIG. 10B. Other processes for obtaininggroup information that add, combine, rearrange, and/or omit one or moresteps described above are possible in accordance with other embodiments.

In accordance with some aspects of the invention, facial recognition ofaudience members in the captured image is performed to provide audienceinformation and in some other aspects, image analysis is used toidentify groups in the audience using image analysis. Processes and datastructures to perform facial recognition by a facial recognition deviceor module in accordance with some aspects of the invention are discussedbelow with reference to FIGS. 11-13. Processes and data structures usedto identify groups in the audience by an image analysis device or modulein accordance with some aspects of the invention are discussed belowwith reference to FIGS. 14-17.

FIG. 11 is a conceptual data structure for a facial recognition recordin accordance with an aspect of the disclosure. A facial recognitionrecord 1100 includes an identifier of the audience member 1105, thereceived facial image 1110, and the facial parameters for facialrecognition comparisons 1115. The identifier may be, for example, a nameand/or nickname of the audience member, or the identifier may be anumber or alphanumeric string that associates the image to a specificaudience member record stored by the content control device or moduleand/or the central control system.

Although an exemplary facial recognition record in accordance withembodiments of the disclosure is described above with reference to FIG.11, other facial recognition records that add, combine, rearrange,and/or omit information are possible in accordance with otherembodiments.

The software and/or hardware modules that perform a facial recognitionprocess in accordance with embodiments of the disclosure is shown inFIG. 12. The facial recognition system 1200 includes a receiving module1205, a facial image identifier module 1210, a facial image analysismodule 1215, a demographic information module 1220 that may generateother information (particularly demographic information), a facialrecognition module 1225, and an audience characteristic module 1230.

The receiving module 1205 receives a captured image and processes thecaptured image to conform the image to the parameters needed to performthe various subsequent processes for facial recognition analysis. Inaccordance with some aspects, the image processing may include, forexample, focus adjustments, color adjustments, edge defining, and otherimage adjustments needed to conform the image to the requirements of thesubsequent modules. In accordance with some aspects, the receivingmodule also receives image information such as, for example, depthinformation, camera information, and lighting information. The receivingmodule 1205 uses the image information in the image processing toconform the image to the required standards.

The processed image is provided to the facial image identifier module1210, which identifies the portions of the image that include a facialimage. The identification may use edge detection and other varioussearch processes to identify those portions of the image that include animage of a face to which facial recognition may be applied. Inaccordance with some aspects, the facial image identifier may alsoperform some image processing to conform the portions including a facialimage to the requirements of an analysis module.

The facial image analysis module 1215 receives the portions of the imagethat include a facial image and performs analysis on each image togenerate the data needed by the other modules to generate theinformation required. For example, the image analysis module maygenerate pixel color and vector data needed to perform edge detection,color detection, and the like needed to perform the various subsequentprocesses. In accordance with some aspects, the facial image analysismodule 1215 also receives the image information and/or a complete imagefor use in performing the analysis. The information generated by thefacial image analysis module 1215 is provided to the demographicinformation module 1220, the facial recognition module 1225, and theaudience characteristic module 1230 to perform the facial recognitionfunction and to generate the demographic and audience characteristicinformation.

The demographic information module 1220 uses the information for eachfacial image received from the facial image analysis module to generatedemographic information for the entire audience, or at least asubstantial portion of the audience (e.g., a representative sample). Thedemographic information may include, for example, the ages,nationalities, races, and the like of the audience members. Thedemographic information may also optionally include a statisticalanalysis of the categories to provide the mean, medium, and otherinformation for each category.

The facial recognition module 1225 receives the information for eachfacial image and compares the information of each facial image to theinformation for the facial images in each facial recognition record todetermine a match and returns the identifier of each record that matchesone of the facial images from the captured image to a predefined degreeof confidence. To facilitate the comparison, the records may includefacial image data that is precomputed to provide quicker comparisons byeliminating the need to analyze each reference image.

The audience characteristic module 1230 receives the information foreach facial image and compiles audience characteristic information. Thecharacteristic information may include the size of the audience, thepositions of the audience in the audience area, and other informationpertaining the physical characteristics of the audience as a whole. Todo so, the audience characteristic module 1230 may also optionallyreceive the image information to help define the spatial characteristicsshown in the image.

Although the above description describes modules of a facial recognitionsystem in accordance with an exemplary embodiment of the disclosure,other facial recognition modules that that add, combine, rearrange,and/or omit modules are possible in accordance with other embodiments.

FIG. 13 illustrates a flow diagram of a process 1300 performed by afacial recognition system to perform facial recognition in a capturedimage of an audience area in accordance with an aspect of thedisclosure. In the process 1300, an image of the audience area isreceived (1305). As discussed above, the received image may be processedto conform the image to the requirements of the process 1300.

Portions of the received (and optionally processed) image that include afacial image are identified (1310). As discussed above, each portion maybe further processed to conform the facial image to the requirements ofthe facial recognition process. A facial recognition comparison to thefacial images stored in the facial recognition record is performed toidentify the records that match the facial images (1315). Theidentifiers of the matching records are provided to the content controlmodule or device.

The information of the facial images from the captured image generatedfor the facial recognition comparisons is used to generate demographicinformation for the audience (1325). The demographic informationprovided is discussed above with respect to FIG. 12. The demographicinformation for the audience is provided to the content control noduleor device (1330).

The information of the facial images from the captured image generatedfor the facial recognition comparisons is also used to generate audiencecharacteristic information (1335). The process for generating theaudience characteristic information and the information generated arediscussed above with reference to FIG. 12. The audience characteristicinformation is also provided to the content control module or device(1340), at which point the process 1300 may end.

An exemplary process for determining audience information using facialrecognition in accordance with embodiments of the disclosure isdescribed above with respect to FIG. 13. Other processes for obtaininggroup information that add, combine, rearrange, and/or omit one or moresteps described above are possible in accordance with other embodiments.

FIG. 14 is a conceptual data structure for a group record 1400 inaccordance with an aspect of the disclosure. The group record 1400includes a group identifier of the group 1405, the received group visualidentifier information 1410, and optionally, the group visual identifierparameters for image comparisons 1415. The identifier 1405 may be, forexample, a name and/or nickname of the group, or the identifier may be anumber or alphanumeric string that associates the group visualidentifier information to a specific group record stored by the contentcontrol device or module and/or the central control system.

Although an exemplary group record in accordance with embodiments of thedisclosure is described above with reference to FIG. 14, other grouprecords that add, combine, rearrange, and/or omit information arepossible in accordance with other embodiments.

An image analysis device or system 1500, comprising software and/orhardware modules that perform an image analysis process in accordancewith some embodiments of the disclosure, is shown in FIG. 15. The imageanalysis system 1500 includes a receiving module 1505, a visualidentifier image module 1510, an image analysis module 1515, ademographic information module 1520, a group recognition module 1525,and an audience characteristic module 1530.

The receiving module 1505 receives a captured image and processes thecaptured image to conform the image to the parameters needed to performthe various subsequent processes for image analysis. In accordance withsome aspects, the image processing may include, for example, focusadjustments, color adjustments, edge defining, and other imageadjustments needed to conform the image to the requirements of thesubsequent modules. In accordance with some aspects, the receivingmodule also receives image information such as, for example, depthinformation, camera information, and lighting information. The receivingmodule 1505 uses the image information in the image processing toconform the image to the required standards.

The processed image is provided to the visual identifier image module1510, which identifies the portions of the image that include a visualidentifier associated with a group and/or audience member, e.g., aparticular color, pattern, or symbol, as described above, that is wornor displayed by all members of the group. The identification may useedge detection and other various search processes to identify thoseportions of the image that include the requisite visual identifier. Inaccordance with some aspects, the visual identifier image module 1510may also perform some image processing to conform the portions includingvisual identifier to the requirements of an analysis module.

The image analysis module 1515 receives the identified portions of theimage and/or the entire captured image, and it performs analysis on theeach identified portion of the image to generate the data from which therequired information is derived. For example, the image analysis module1515 may generate pixel color and vector data needed to perform edgedetection, pixel color detection, and the like needed to perform thevarious subsequent processes. In accordance with some aspects, the imageanalysis module 1515 also receives the image information and/or acomplete image for use in performing the analysis. The informationgenerated by the image analysis module 1515 is provided to thedemographic information module 1520, the group recognition module 1525,and the audience characteristic module 1530 for use in performing grouprecognition and to generate the demographic and audience characteristicinformation.

The demographic information module 1520 uses the information from eachidentified portion and/or the entire image received from the imageanalysis module 1515 to generate demographic information for the entireaudience, or at least a substantial portion of the audience (e.g., arepresentative sample). The demographic information may include, forexample, the ages, nationalities, races, and the like of the audiencemembers. The demographic information may also optionally include astatistical analysis of the categories to provide the mean, medium, andother information for each category.

The group recognition module 1525 receives the information from thereceived image and compares the information of the group visualidentifier information in each group record to determine a match andreturns the group identifier of each group record that matches the datafrom the captured image to a predefined degree of confidence. Tofacilitate the comparison, the records may include visual identifierimage data that is precomputed to provide quicker comparisons byeliminating the need to analyze each reference image.

The audience characteristic module 1530 receives the information for thecaptured image and compiles audience characteristic information. Thecharacteristic information may include the size of the audience, thepositions of the audience in the audience area, and other informationpertaining the physical characteristics of the audience as a whole. Todo so, the audience characteristic module 1530 may also optionallyreceive the image information to help define the spatial characteristicsshown in the image.

Although the above description describes modules of an image analysisdevice or system in accordance with an exemplary embodiment of thedisclosure, other image processing modules that that add, combine,rearrange, and/or omit modules are possible in accordance with otherembodiments.

FIG. 16 illustrates a flow diagram of a process 1600 performed by animage analysis device or module to detect groups of audience members ina captured image of an audience area in accordance with an aspect of thedisclosure. In the process 1600, an image of the audience area isreceived (1605). As discussed above, the received image may be processedto conform the image to the requirements of the process 1600.

The captured image is analyzed to detect groups in the audience based onthe group identifier information in the group records (1610). Inaccordance with some embodiments, the analysis may include determiningthe color of each pixels and a total amount of pixels for each color inthe captured image. The pixel colors may be ranked based on the numberof pixels of each color in the image. The pixel colors present arecompared to the colors identified in the visual identifier informationof each group record. Other processes for analyzing the image inaccordance with some other embodiments of the invention are discussedbelow with reference to FIGS. 17 and 18. The groups present are thendetermined by whether a threshold amount of pixels of the identifiedcolor is present and/or by the rankings of the pixel colors. The groupidentifier from each group record that has a match for the group visualidentifier information is provided to the content provider device ormodule as part of the audience information (1615), and the process 1600may end.

An exemplary process for determining audience information using imageanalysis in accordance with embodiments of the disclosure is describedabove with respect to FIG. 16. Other processes for obtaining groupinformation that add, combine, rearrange, and/or omit one or more stepsdescribed above are possible in accordance with other embodiments.

FIG. 17 illustrates a process 1700 for analyzing the captured image ofthe audience area for visual identifiers associated with each group inaccordance with some aspects of the disclosure. Portions of the received(and optionally processed) image that include a visual identifier areidentified (1705). As discussed above, each identified portion may befurther processed to conform the requirements of the analysis process.An image comparison to the visual images stored in the group identifierinformation in each group record is performed to identify the grouprecords that match the visual identifiers in the identified portions ofthe image (1710). The process 1700 may then end, and the above-describedprocess 1600 (FIG. 16) may continue and provide the group identifiers ofthe matching group records to the content control module or device, asdiscussed above.

An exemplary process for performing image analysis in accordance withsome embodiments of the disclosure is described above with respect toFIG. 17. Other processes for obtaining group information that add,combine, rearrange, and/or omit one or more steps described above arepossible in accordance with other embodiments.

FIG. 18 illustrates a process 1800, in accordance with some embodiments,for analyzing the captured image of the audience area for visualidentifiers (e.g., a particular color, pattern or symbol, as discussedabove) associated with each group. Portions of the received (andoptionally processed) image of audience area that may include a visualidentifier are identified (1805). For example, the process may identifyportions of the image including a badge, pin, lanyard, flag, article ofclothing, or some other accessory that is worn or displayed by anaudience member. As discussed above, each identified portion may befurther processed to conform the requirements of the analysis process.

Color recognition may be performed on each identified portion todetermine the color(s) of the visual identifier (1810). The color(s) arethen used to determine a group associated with each portion based on thecolors stored in the group visual identifier information of each grouprecord (1815). The group identifier of each group identified is added tothe audience information (1820). In accordance with some embodiments, acount of the number of portions identified to be associated with eachidentified group may be determined and added to the audience informationfor use in determining the media content provided based on the number ofaudience members in each group. The process 1800 may then end, and theabove-described process 1600 (FIG. 16) may continue and provide thegroup identifiers of the matching group records to the content controlmodule or device as discussed above.

An exemplary process for performing image analysis using colorrecognition of visual identifiers in accordance with some embodiments ofthe disclosure is described above with respect to FIG. 18. Otherprocesses for obtaining group information that add, combine, rearrange,and/or omit one or more steps described above are possible in accordancewith other embodiments.

FIG. 19 shows a schematic of an alternative embodiment of the invention,having an image-capturing device 1910 that captures images, and a mediacontent playback device 1920 that plays media content. Whileimage-capturing device 1910 could be a camera that captures individualimages, the image-capturing device 1910 is preferably a camera thatcaptures a stream of images and saves those images to a memory, forexample a hard drive or RAM in a computer system 1912. Preferably, animage stream is saved on the memory as a loop, for example a ten-minuteloop or an hour-long loop, which is constantly overwritten as the numberof images stored in the memory reach a memory threshold. In someembodiments, the image-capturing device 1910 could activate to capture aset of images in response to a trigger, for example a motion-sensor (notshown) electronically coupled to the camera or an inductive vehicle loopsensor, to capture a set of images.

Preferably, after a set of image fingerprints has been derived from aset of images, the set of images is deleted or overwritten in memory. Insome embodiments, the image-capturing device 1910 may capture just asingle image, but preferably the image-capturing device 1910 captures aplurality of images over time. In exemplary embodiments, theimage-capturing device 1910 is controlled by an image computer system1912 having software that, when executed by a processor, capturesimages, saves at least some of them in a transient or non-transientmemory (not show), and forwards them to a server 1950 over a network1960 for analysis. The network 1960 is shown symbolically in FIG. 19 asa “cloud” network that electronically connects devices with one another,such that data can be transmitted from one device to another via thenetwork 1960. The server 1950 then analyzes one or more images sent fromthe image computer system 1912 to determine what sort of media contentis appropriate as a function of data gleaned from the image-capturingdevice 1910. As previously mentioned, computer-system 1912 and server1950 are shown symbolically here as two discrete computer systemselectronically coupled using the network 1960, but they could beintegrated into a single computer system in some embodiments. Likewise,as previously mentioned, while the image-capturing device 1910 is shownas a single device, a plurality of image-capturing devices could beelectronically coupled to the image-computer system 1912 or to theserver 1950.

The image-capturing device 1910 is preferably configured to captureimages in real-time, but it could be configured to continuously recordsets of images, for example by using a looped memory to hold a videostream. Such embodiments could be preferred in situations where animage-capturing device is monitoring a road with constant traffic, or aparking lot with multiple customers. In embodiments in which images areanalyzed in real-time, extracted image fingerprints may be generated, toprevent raw images from being saved on a transient memory. In otherembodiments, the image-capturing device 1910 is configured to captureimages in response to a trigger from a sensor electronically coupled toimage-capturing device 1910. For example, a motion sensor could triggercapturing of an image, or a detection of a vehicle within an inductivevehicle loop sensor could trigger capturing of an image. In suchembodiments, the image-capturing device 1910 preferably captures a setof images over time, for example a video stream that starts when amotion sensor detects movement, and ends when the motion sensor fails todetect a movement within a threshold period of time. Feedback mechanismscould also be introduced that trigger alterations in the wayimage-capturing device 1910 captures images. For example,image-capturing device 1910 could be configured only to derive facialimage fingerprints using a real-time analysis algorithm that comparesderived facial image fingerprints against facial image fingerprints inan image database 1942, and recognition of matching image fingerprintscould trigger image-capturing device 1910 to derive both facial imagefingerprints and transactional identifiers, or both facial imagefingerprints and interactions. While a single image-capturing device1910 is shown, a media content system could have any number ofimage-capturing devices that work in conjunction with one another withany number of computer systems.

Server 1950 performs its analysis of a set of images using one or moredatabases, such as the image database 1942, a relationship database1944, an attribute database 1946, and a media database 1948. While eachof these databases is shown symbolically as a separatenetwork-attached-storage (NAS) database electronically coupled tonetwork 1960, each of the databases could be integrated into a singlememory, or could be distributed among a plurality of memory systemsdepending upon need. The databases could be spread among a plurality ofsystems, and may be integrated in any suitable manner.

The image database 1942 is used to associate image fingerprints withidentifiers. Preferably, the image database 1942 is pre-populated withidentifiers that are entered into the database by an authenticated useror by importing such data from a third-party database, such as athird-party database 1934. Authenticated users could compriseadministrators of the system, or guests with limited rights, for examplea teacher who is provided rights to add identifiers for students in herclassroom (e.g. a student ID). Such identifiers could be added using anysuitable user interface, for example a website hosted by a server 1950,an email address that automatically processes filled form templatescontaining identifier information, a service kiosk (not shown), or acombination thereof. In other embodiments, the server 1950 could querythe image database 1942 for identifiers that match a given imagefingerprint, and if the query returns a null result, the server 1950could then add a unique identifier into the image database 1942. Forexample, a facial fingerprint could be searched, and if the imagedatabase 1942 returns a null result, that facial fingerprint could beassigned a unique identifier for the future. In this manner, as themedia content system is used over time, the image database 1942 couldgrow dynamically.

The relationship database 1944 is used to associate identifiers withother identifiers. Again, preferably, the relationship database 1944 ispre-populated with relationships that are entered into the database byan authenticated user or by importing such data from a third-partydatabase, such as the third-party database 1934, but the media contentsystem could dynamically add to the relationship database 1944 as well.

In some embodiments, when the server 1950 identifies image fingerprintsbelonging to commonly associated classes, and the relationship database1944 returns a null value, the server 1950 could create a newrelationship. For example, if the server 1950 detects a group of facialfingerprints and a car license plate within a set of images, anddiscovers that identifiers for the people with those facial fingerprintsand the identifier for the car with that license plate are not in therelationship database 1944, the server 1950 could then add arelationship to the relationship database 1944 to associate theidentifiers of those people with that car. In some embodiments,identifiers captured within a set of images can alter relationshipswithin relationship database 1944. For example, every time a set ofimages contains both a person identifier and a car identifier, a weightof a relationship of occupant between the person identifier and the caridentifier could be altered (e.g. an increment of a value). On the otherhand, every time a set of images contains a car identifier with adifferent person identifier, a weight of a relationship of occupantbetween the car identifier and other person identifiers could bedecremented. In this way, as the system repeatedly detects the sameoccupants in a car within sets of images, the weight of the relationshipbetween the occupants and the car can be incremented, and as systemrepeatedly detects occupants in a car within sets of images that are ofnot of a person that has an existing relationship with the car can havethat relationship weight decremented.

In some embodiments, several relationships could be defined betweenidentifiers, for example an association of a person with a vehicle, andan association of a car seat of that vehicle (e.g. driver's seat, frontpassenger seat, middle rear seat). In some embodiments, mere proximityis enough to create an association (e.g. a group of identifiers withinthe same set of images), while in other embodiments motion recognitionis necessary (e.g. a movement of a body associated with a facialfingerprint moving into the front passenger door of a vehicle) totrigger addition of a new relationship to the relationship database1944.

The attribute database 1946 is used to associate an identifier withattributes of that identifier. Any suitable attribute could be used, forexample a preference of a good/service or a demographic. Again, whilethe attribute database 1946 is preferably pre-populated with attributesthat are entered into the database by an authenticated user or byimporting such data from a third-party database, such as the third-partydatabase 1934, the media content system could dynamically add to theattribute database 1946. In some embodiments, the media content systemcould analyze activities that are captured in one or more sets ofimages. Such embodiments preferably retrieve sets of image data fromadditional image-capturing devices controlled by the server 1950 or bytheir own image computer systems, such as the image computer system1912, and could also glean data from third party sensors, such as athird-party sensor 1932, which feeds information into the third-partycomputer system 1930 to be stored on the third-party database 1934.

For example, in one embodiment, a set of images could capture an imageof a person wearing a branded piece of clothing. The server 1950 couldthen analyze that set of images to uniquely identify a facialfingerprint that is associated with an identifier of a person, anduniquely identify a logo fingerprint that is associated with a brand anda clothing type (e.g. a brand Nike® and a clothing type of a sportsbra). The server 1950 could then create an attribute for that person'sidentifier in the attribute database 1946 to associate that person'sunique identifier with the brand and/or the clothing type. Attributescould be weighted in any suitable manner, for example via third-partydata (e.g. updated data retrieved from a server that contains updatedinformation for a person's attributes), and/or via analysis of sets ofimages over time (e.g. a person who is found holding a tennis racket in25 sets of images and who is found holding a basketball in 3 sets ofimages could have an associated weight distribution favoring a tennisracket, using an algorithm that uses the number of images matching anattribute as an input of the function).

In other embodiments, interactions that are analyzed within a set ofimages could be used to alter attributes in the attribute database 1946.For example, if the image-capturing device 1910 captures a set of imagesof an identified person watching media content that is associated with atennis racket, and identifies an action of that person looking away fromthe media content or identifies that the person has not looked at themedia content within a threshold period of time, the server 1950 coulddecrement the weight of the tennis racket attribute associated with thatperson's identifier in the attribute database 1946. Interactions couldalso include body movements of an analyzed entity. For example, a smileor an extended thumbs-up could indicate an increased favorabilitytowards another object in the image, while a frown or an extendedthumbs-down could indicate a decreased favorability towards anotherobject in the image. In response to detecting such interactions in a setof images along with an identified object or transaction identifier inthe set of images, the server 1950 could be configured to alter anattribute of the attribute database 1946, for example by incrementing ordecrementing a weight of a numerical attribute value associated with anidentifier of the identified object (e.g. a detected smile when holdinga type of identified food increments a weight of a value associated withthat type of identified food).

The media content database 1948 is used to associate one or moreattributes with a set of media content to provide targeted media contentto the media content playback device 1920. Again, while the mediacontent database 1948 is preferably pre-populated with media contentthat is saved onto the database by an authenticated user or by importingsuch data from a third-party database, such as the third-party database1934, the media content system could dynamically add new media contentto the media content database 1946, or could even add new associationsto the media content database 1946. In preferred embodiments, newassociations could be added as a function of identified triggeringinteractions within a set of images. For example, if the image-capturingdevice 1910 captures a set of images of an identified person watchingmedia content that is associated with an ice cream cone for a thresholdperiod of time, the server 1950 could identify that action and couldthen associate one or more attributes of that identified person to themedia content. Preferably, such additions are only triggered with rareidentified triggers and/or interactions, for example a movement towardsthe media content playback device 1920, or a large number of personshaving a common attribute (e.g. a smile while consuming media content, athreshold period of time looking in a particular direction).

The server 1950 could then transmit one or more sets of media contentfrom the media content database 1948 to a playback computer system 1922for playing on the media content playback device 1920. In someembodiments, the set of media content is weighted in accordance withweighted attributes of the identifiers, such that media content withgreater weight is afforded priority. In other embodiments, media contentfrom the set of media content is randomly chosen for playback. In someembodiments, both the media content and attributes are sent to theplayback computer system 1922, where the attributes alter the manner inwhich selected media content is played. For example, an attribute may bea volume of a video that is played, or may be a language that an audioportion of a video is played in, or may be a language that in which avideo's subtitles are shown. In some embodiments, attributes may triggerdifferent media content to be derived, for example an age attribute maytrigger a decision between a cartoon or a live-action media file.

While FIG. 19 illustrates a plurality of computer systems that workcooperatively with one another—the image computer system 1912, theplayback computer system 1922, the server 1950, and the third-partycomputer system 1930—any number or combination of computer systems couldbe used. For example, the image-capturing device 1910 and the mediacontent playback device 1920 could be controlled by a common computersystem, and that common computer system could even function in the roleof the server 1950 with one or more of the databases 1942, 1944, 1946,and 1948 stored on its local memory. In other embodiments, the roles andresponsibilities of each of the computer systems could be distributedamong a plurality of computer systems.

FIG. 20 shows an exemplary process 2000 that could be used by one ormore media content computer systems to provide media content to a mediacontent playback device. In step 2005, the media content computer systemcould receive a set of images, for example from an image capture device.In step 2010, the computer system could analyze the set of images toderive a set of image fingerprints, such as facial fingerprints, licenseplates, or shirt tags. In step 2015, the computer system could generatea first set of image identifiers, for example by cross-referencing theset of image fingerprints against in image database comprisingcorrelations between image fingerprints and identifiers. In step 2025,the computer system could generate a second set of identifiers using atleast some, if not all, of the first set of identifiers, for example byquerying a relationship database using the first set of identifiers. Instep 2030, the computer system could generate a set of attributes usingat least some of the second set of identifiers, and in some embodimentsusing at least some of both the second set of identifiers and the firstset of identifiers. Derivation of the set of attributes could beperformed, for example, by querying an attribute database comprisingcorrelations between identifiers and attributes. In step 2035, thecomputer system could generate a set of media content using the set ofattributes, for example by querying a media content database comprisingcorrelations between attributes and media content. Preferably theattributes are weighted, where some attributes have heavier weights thanothers. In some embodiments, the set of media content could comprisejust identifiers that are used by a playback system to retrieve mediacontent, while in other embodiments the set of media content couldcomprise media files themselves. Preferably, the set of media content isweighted or organized into a weighted list, where media content with ahigher weight has a higher correlation with higher weighted attributes.In step 2040, the computer system could then play at least some of theset of media content on a media content playback device.

The embodiments herein could be utilized to provide targeted mediacontent to a media content playback device using any suitable identifiedrelationship within a set of images. For example, the image-capturingdevice 1910 could capture an image of a face, and the image computersystem 1912 could transmit that image or a facial fingerprint via thenetwork 1960 to the server 1950 for analysis. The server 1950 could thencompare that facial fingerprint via the network 1960 with the imagedatabase 1942 to determine an identifier. That identifier is then usedto query the attribute database 1946, to find out that the identifiedperson has a preferred language, e.g., “Dutch.” The server 1950 thenchecks the “Dutch” attribute against the media database 1948 to findmedia content preferably watched by someone who is Dutch, which is thenforwarded to the playback computer system 1922. The playback computersystem 1922 then pulls one of the media content files and plays it onthe media content playback device 1920 of an A/V display at a servicestation fuel pump, checking subsequent images captured by theimage-capturing device 1910 to see if the person is watching. If theperson is not watching within a certain threshold time period, theplayback computer system 1922 then cycles to another media file.

In another embodiment, the image-capturing device 1910 captures a groupof people and transmits a set of images to the server 1950 for analysis.The server 1950 identifies a first set or number (e.g., 20) of the groupof people as wearing an aqua-colored, square-shaped tag, and a secondset or number (e.g., 5) as wearing a teal-colored, square-shaped tag. Inthis embodiment, the server 1950 is configured to only analyzesquare-shaped tags. The server 1950 then compares “aqua” against theimage database 1942 to determine that “aqua” is associated with a firstidentified group, and compares “teal” against the image database 1942 todetermine that “teal” is associated with a second identified group.These identifiers are then compared against the attribute database 1946to determine the attribute “English” is associated with the firstidentified group, and that the attribute “Spanish” is associated withthe second identified group. “English” is given a weight of 20, sincethere were 20 instances of aqua found in the set of images, while“Spanish” is given a weight of 5, accordingly. Those attributes are thencompared against the media database 1948 to retrieve media havingEnglish audio and Spanish subtitles. The playback computer system 1922then plays that media on the media content playback device 1920 of amuseum exhibit after receiving that media content from the server 1950.

In another embodiment, the image-capturing device 1910 may be configuredto capture a group of people entering a vehicle, and to transmit a setof images to the server 1950 for analysis. The server 1950 would thenquery the image database 1942 to determine identifiers for the peopleand the vehicle, and then query the relationship database 1944 todetermine if those people are associated with the vehicle. If the latterquery results in a null value, the server 1950 could then addrelationships between the identifiers of the people and the identifiersof the vehicle to the relationship database 1944. This relationshipcould be set to have no expiration date, or it could be set to expire atthe end of a given time period, for example, at the end of a springbreak holiday week or on Sunday when the weekend is over. At a latertime, the image-capturing device 1910 could then capture an image ofjust that vehicle, and transmit that set of images to the server 1950for analysis. The server 1950 would then query the image database 1942to determine an identifier for the vehicle using the captured licenseplate number, and then could query the relationship database 1944 todetermine a relationship between the identifier of the vehicle andvarious identifiers for the people. The server 1950 could then query theattribute database 1946 using the identifiers of the people to determineattributes of the people, and could weight all the attributesappropriately in an aggregated list of attributes, for example by addingthe attributes together (e.g. person 1 has a 0.2 attribute for ahamburger and person 2 has a 0.3 attribute for a hamburger resulting inan aggregated 0.5 attribute for a hamburger). The server 1950 could thenquery the media content database 1948 to determine a set of mediacontent appropriate for the attributes in that vehicle, and could sendthat set of media content to the playback computer system 1922, whichwould be coupled to a media content playback device 1920 of a billboardthat will display to that vehicle. In some embodiments, that media couldbe played immediately, while in other embodiments, the system could beprogrammed to play the media at a given distance (e.g., one mile or 1.5km) up the road, triggered as a function of a calculated time/distancethreshold, or when a separate image-capturing device placed just beforethe billboard identifies the same license plate.

In other embodiments, groups of vehicles could be analyzed to deriveattribute data for the group. For example, a set of images of variousvehicles could be captured by image-capturing device 1910 over a periodof time (e.g. a single instant or a period of five seconds). Imagecomputer system 1912 could then analyze the set of images to generateimage fingerprints. In some embodiments the analysis could generateidentifiers of the cars, such as a license plate of a vehicle or amake/model of the vehicle, while in other embodiments the imagefingerprints are transmitted to server 1950 to query image database 1942to determine identifiers of each vehicle. The generated identifierscould then be used by server 1950 to query relationship database 1944and glean a plurality of related identifiers (e.g. occupants of thosevehicles or types of vehicles), which could, in turn, be used to queryattribute database 1946 and glean attributes for each relatedidentifier. In this manner, attributes for a large population of derivedvehicle occupants could be gleaned. Preferably, the attributes areaggregated, for example by summing the weighted values of theattributes. While the totality of the attributes could then be used byserver 1950 to derive sets of media content for media database 1948, inpreferred embodiment only a subset of the attributes are selected,preferably as a function of the weight of the attributes (e.g. the topattribute or the top 5 attributes), to be used to derive sets ofappropriate media content from media database 1948. Server 1950 couldthen transmit the set of media content to playback computer system 1922to play to media content playback device 1920. In some embodiments,where the vehicles are not moving, for example a traffic jam or at adrive-in movie theater, the playback computer system 1922 could beconfigured to avoid repeating media (e.g. by deleting media content froma list of the set of media saved on memory after playback), ornon-repeating media within a threshold of time (e.g. within a half anhour).

In some embodiments, a third-party computer system 1930 could beprovided access to relationship database 1944 to verify relationshipdata that has been saved to relationship database 1944. For example,server 1950 could derive both a driver identifier (e.g. by recognizingan interaction of entering a driver's door of a vehicle and capturing aone or more images of the entity entering that vehicle) and a vehicleidentifier from a set of images (e.g. by querying image database 1942with a facial fingerprint and a license plate number) and could thensave both identifiers as a relationship along with a time-stamp inrelationship database 1944. A third-party computer system 1930 couldthen transmit a query to the server 1950 using an identifier of avehicle (e.g. a license plate of the vehicle) and a time period and theserver 1950 could respond to the third-party sensor by providingidentifiers of all parties that have been captured drivers of thatvehicle within the time period. In other embodiments, the third-partycomputer system 1930 could provide a facial fingerprint or an identifierof an authorized driver, and server 1950 could be programmed to onlyprovide information that an entity that is not the authorized driver oris not a person who matches the received facial fingerprint is thedriver of the car.

In some embodiments, third-party computer system 1930 could beconfigured to have authorized access to set a triggering alarm on server1950 related to a vehicle identifier. For example, third-party computersystem 1930 could provide a vehicle identifier to server 1950 along witha set of facial fingerprints, and server 1950, in response, could thencreate identifiers for each of the set of facial fingerprints in imagedatabase 1942, and could associate the vehicle identifier with the newlycreated identifiers in relationship database 1944 with an “authorizeddriver” tag. Server 1950 could then analyze sets of images for facialfingerprints of entities having a recognized interaction of driving avehicle, or exiting/exiting a driver's door of a vehicle, and forvehicle fingerprints (e.g. a license plate number). Server 1950 couldthen derive identifiers for entities having the driver interaction withthe vehicle and the identifier of the vehicle, and search relationshipdatabase 1944 for a relationship of “authorized driver” between theinput search query terms. If server 1950 fails to find an authorizeddriver relationship, then server 1950 could then transmit to third-partycomputer system 1930 (or any other computer system defined by a user ofthird-party computer system 1930) an alert that an unauthorized driveris driving the vehicle.

In a related embodiment, server 1950 could be configured to save one ormore sets of images from image-capturing device 1910 to a memory inresponse to a trigger, for example local memory on server 1950, mediadatabase 1948, or third-party database 1934. In some embodiments,third-party computer system 1930 could transmit a request for server1950 to save images having or lacking a relationship, or having orlacking an attribute. For example, server 1950 could be configured tosave one or more sets of images where server 1950 fails to find anauthorized driver relationship between a captured driver and anidentified vehicle. In another embodiment, server 1950 could beconfigured to save one or more sets of images where server 1950identifies an attribute of “save image” with an identifier. Such imagescould be saved with metadata information, such as a time-stamp, alocation of the image-capturing device, and/or an identifier of alocation of the image-capturing device, and could be pushed to athird-party computer system that requested the information in real-time,or could be stored on a memory for retrieval at a later time. In someembodiments, the request from third-party computer system 1930 couldhave an expiration-date timer to delete sets of images after a providedtime period threshold.

In other embodiments, server 1950 could transmit one or more attributesto a third-party computer system 1930 instead of, or in addition to,playing a set of media content. For example, where image-capturingdevice 1910 captures images about a gas station pump and media contentplayback device 1920 is a screen at a gas station, third-party computersystem 1930 could be configured to control an auto-select feature of agas pump. In such embodiments, the set of captured images fromimage-capturing device 1910 could comprise a vehicle fingerprint, suchas a photo of a license plate, which can be processed by a computersystem, such as image computer system 1912 or server 1950 to derive alicense plate identifier or a make/model vehicle identifier. Server 1950or image computer system 1912 could then use that derived identifier toquery attribute database 1946 and determine a preferred gas type of thevehicle, and could then transmit that preferred gas type to third-partycomputer system 1930, which then auto-selects the preferred gas type.

FIG. 21 illustrates an alternative system 2100 for providing mediacontent to supplement an exhibit in accordance with an aspect of thedisclosure. The system 2100 includes a facial recognition module 2102; acentral control module 2104; and exhibit control modules 2108 and 2112that are communicatively connected by a network 2110. A portablepersonal communication device 2120 and a computer 2125 may also beconnected to the network 2110. Although shown as four separate devicesor functionalities in FIG. 21, the facial recognition module 2102; thecentral control module 2104; and the exhibit control module 2108 and2112 may be provided by a single computing system. Alternatively, theprocesses that provide one or more of the facial recognition module2102, the central control module 2104, and the exhibit control module2108 may be distributed across multiple systems that are communicativelyconnected via the network 2110.

The facial recognition module, functionality, or system 2102 may beimplemented or functionalized by a computer system that includes amemory and a processing unit to perform the processes for providingfacial recognition and/or other image analysis. The computer system thatimplements the facial recognition module, functionality, or system mayinclude one or more servers, routers, computer systems, and/or memorysystems that are communicatively connected via a network to providefacial recognition and/or other image analysis.

The central control module 2104 may be implemented or functionalized bya computer system that includes a memory and a processing unit toperform processes for storing and providing media content for one ormore exhibits in a venue. The central control module 2104 may alsoadvantageously store and update location information of a user. Thecentral control functionality may be provided by a central controlsystem for the venue. Specifically, the central control module 2104 maybe implemented or functionalized by a system that includes one or moreservers, routers, computer systems, and/or memory systems that arecommunicatively connected via a network to store and provide mediacontent for one or more exhibits in the venue, as well as to store andupdate location information for registered users.

Each of the exhibit control modules 2108 and 2112 may be implemented orfunctionalized by a computer system that controls devices in the exhibitarea that include an image capture device. Advantageously, one computersystem may control devices for more than one exhibit. In specificembodiments, the exhibit control modules 2108 and 2112 may beimplemented or functionalized by a system that includes one or moreservers, routers, computer systems, memory systems, an image capturedevice and/or media playback devices that are communicatively connectedvia a local network to obtain and present media content for the exhibit.

The network 2110 may advantageously be the Internet. Alternatively, thenetwork 2110 may be a Wide Area Network (WAN), a Local Area Network(LAN), or any combination of Internet, WAN, and LAN that can be usedcommunicatively to connect the various devices and/or modules shown inFIG. 21.

The portable personal communication device 2120 may a smart phone,tablet, Personal Digital Assistant (PDA), a laptop computer, or anyother device that is connectable to the network 2110 via wirelessconnection 2122. The computer 2125 may advantageously connect to thenetwork 2110 via either a conventional “wired” or a wireless connection.The computer 2125 may be, for example, a desktop computer, a laptop, asmart television, and/or any other device that connects to the network110. The portable personal communication device 2120 and/or the computer2125 allow a user to interact with one or more of the above-describedmodules to provide information such as, for example, personalinformation to be added to registered user information of the user.

Although a particular system of devices and/or functional modules isdescribed above with respect to FIG. 21, other system architecturesthat, add, remove, and/or combine various devices and/or modules may beused to perform various processes in accordance with various otheraspects of the disclosure.

FIG. 22 is a high-level block diagram showing architecture of anexemplary processing system 2200 that may be used according to someaspects of the disclosure. The processing system 2200 can represent acomputer system that provides a facial recognition functionality, acentral control functionality, an exhibit control functionality, and/orother components or functionalities. Certain standard and well-knowncomponents of a processing system which are not germane to the subjectmatter of this disclosure are not shown in FIG. 22.

Processing system 2200 includes one or more processors 2205 in operativecommunication with memory 2210 and coupled to a bus system 2212. The bussystem 2212, as shown in FIG. 22, is a schematic representation of anyone or more separate physical buses and/or point-to-point connections,connected by appropriate bridges, adapters and/or controllers. The bussystem 2212, therefore, may include, for example, a system bus, aPeripheral Component Interconnect (PCI) bus, a HyperTransport orindustry standard architecture (ISA) bus, a small computer systeminterface (SCSI) bus, a universal serial bus (USB), or an Institute ofElectrical and Electronics Engineers (IEEE) standard 1394 bus (sometimesreferred to as “Firewire”).

The one or more processors 2205 are the central processing units (CPUs)of the processing system 2200 and, thus, control its overall operation.In certain aspects, the one or more processors 2205 accomplish this byexecuting software stored in memory 2210. The processor(s) 2205 may be,or may include, one or more programmable general-purpose orspecial-purpose microprocessors, digital signal processors (DSPs),programmable controllers, application specific integrated circuits(ASICs), programmable logic devices (PLDs), or the like, or acombination of such devices.

Memory 2210 represents any form of random-access memory (RAM), read-onlymemory (ROM), flash memory, or the like, or a combination of suchdevices. Memory 2210 includes the main memory of the processing system2200. Instructions 215 implementing the process steps of described belowmay reside in memory 2210 and are executed by the processor(s) 2205 frommemory 2210.

Also advantageously connected operatively to the processor(s) 2205through the bus system 2212 are one or more internal or external massstorage devices 2220, and a network interface 2222. The mass storagedevice(s) 2220 may be, or may include, any conventional medium forstoring large volumes of data in a non-volatile manner, such as one ormore solid state, magnetic, or optical based disks. The networkinterface 2222 provides the processing system 2200 with the ability tocommunicate with remote devices (e.g., storage servers) over a network,and may be, for example, an Ethernet adapter, a Fiber Channel adapter,or the like.

The processing system 2200 also advantageously includes one or moreinput/output (I/O) devices 2217 operatively coupled to the bus system2212. The I/O devices 2217 may include, for example, a display device, akeyboard, a mouse, etc.

FIG. 23 illustrates a venue with multiple exhibits or attractions inaccordance with an aspect of the invention. Venue 2300 includes multipleexhibits or attractions including exhibits 2305, 2310, 2315, 2320, and2325. Each exhibit has an associated image capture device 2307, 2312,2317, 2322, and 2327 respectively.

For example, exhibit 2305 has an associated image capture device 2307that is focused on an area in front of exhibit 2305 to capture an imagewith facial images of patrons entering the exhibit 2305. Preferably, thearea is defined by an admin user who focuses on sections of an image, orcontrols a camera to point at pre-defined areas in front of exhibit2305. Preferably, the image capture device is focused on an area near anentrance and/or exit of the exhibit to ensure capturing images with thegreatest number of facial images of patrons. Furthermore, although oneassociated image capture device is shown with each exhibit, an exhibitcan have more than one associated image capture devices in accordancewith various aspects of this disclosure. Furthermore, image capturedevices, such as image capture device 2328, may be placed in variouslocations around the venue 300, such as but not limited to, high trafficareas to capture location information of patron in the location and/oras many patrons as possible.

In accordance with some aspects, a proximity sensor 2329 may beassociated with an image capture device 2328 to detect when one or morepatrons are in a location. The proximity sensor may send a signal to theimage capture 2328 when one or more patrons are detected. The imagecapture device then captures an image of the location for use inlocation determination of registered users as will be discussed in moredetail below.

Although an exemplary venue in accordance with an aspect of theinvention is described above with reference to FIG. 23, otherconfigurations that add, remove, combine, and/or move componentsrelative to one another are possible.

FIG. 24 illustrates a flow diagram of a process for providing locationinformation about users at various locations in a venue in accordancewith an aspect of the disclosure. One example of a registration processfor users is described below in detail with respect to FIG. 36. Theprocess 2400 captures images of patrons in various known locationsthroughout the venue 2405 using a plurality of image capture devices,for example devices 2307, 2312, 2317, 2322, 2327, and 2328 in FIG. 23.For example, a location may be, but is not limited to, an entrance or anexit of an exhibit; an entrance or exit of the venue; and/or other hightraffic locations in the venue. The captured images are advantageouslyprovided to a facial recognition device or module (e.g. 2102 in FIG. 1)in step 2410. The facial recognition device or module identifies thedesired portions of each captured image that include the facial image ofone or more patrons in step 2415.

A facial recognition process is then performed on the identified portionof each captured image to identify (e.g., with appropriate userinformation) each registered user in each image and associate eachidentified user with the location where the image was captured in step2420. The location of each identified user is then updated in a locationrecord, such as those shown in FIG. 33, for the user in step 2425. Crowdconditions in a location including, but not limited to, crowd size foran exhibit, may also be determined from the image of each location instep 2430. For example, software could count entities with unique facialfingerprints in a set of images taken over time. The crowd informationfor each location is then updated in a location record, for example indatabase 3400 shown in FIG. 34, in step 2435 after which the process2400 ends.

As discussed above, an exhibit control system, device, or module (e.g.2108, 2112, FIG. 21) could capture the images of the patrons in near anexhibit/high traffic area. In some embodiments, the exhibit controlsystem, device, or module may be a separate system that is notassociated with a specific exhibit. Instead, the image capture device(e.g. 2328, FIG. 23) is a high traffic location and associated with alocation control system and/or associated exhibit control system, tosend information to the central control device or module (e.g. 2104,FIG. 21).

The above describes an overall process for obtaining locationinformation registered users and/or crowd information for a location ina venue in accordance with one aspect of the disclosure. However, otherprocesses that add, combine, remove, and/or reorder the steps of theprocess are possible.

FIG. 25 is a block diagram of components of an exemplary exhibit controlsystem, device or module 2500 and/or location control system (similar to2108, 2112, FIG. 21) which, in accordance with an aspect of thedisclosure, includes a controller 2505 and an image capture device 2515(similar to 2328, FIG. 23).

The controller 2505 may be implemented as a processing system thatcontrols the image capture device 2515 in capturing images of an areaand/or location to obtain the location information for registered usersusing image analysis of the captured image. In accordance with someaspects, the controller 2505 may also control one or more components ofthe exhibit. These components may include, for example, valves,hydraulic lifts, animatronics that provide motion in the exhibit, andany other components that receive instructions to perform a task tofacilitate the presentation of the exhibit. In some other aspects, thecontrol system for more than one exhibit may be provided by a processingsystem. These control devices could be controlled via a triggeringmechanism that is triggered by recognition of pre-defined imagecharacteristics or movements within a set of images. For example, atable could define that an animatronic activates when a predefinedteacher or a predefined student or an entity having the attribute of ateacher is recognized within a set of images.

The image capture device 2515 may be a camera that captures still imagesand/or a video camera that captures video images (i.e. a set of images).In the exemplary embodiment shown in FIG. 25, the image capture device2515 is a separate device including a processing system that iscommunicatively connected to the controller 2505 via a wireless or wiredconnection. In some other aspects, the image capture device 2515 is anI/O device of the processing system or module including the controller2505. As discussed above, the image capture device 2515 is positionedsuch that the device is focused on a particular area near the exhibit ina manner to capture images that include facial images of patrons in thearea. The image capture device 2515 may also capture, record, orotherwise provide other information, such as depth information for theimaged image objects.

Although an exemplary embodiment of an exhibit control device or moduleis described above with respect to FIG. 25, other embodiments that add,combine, rearrange, and/or remove components are possible.

FIG. 26 illustrates a flow diagram of an exemplary process 2600performed by the exhibit control device or module (2500 or 2108/2112) tocapture images to provide location information for registered users inaccordance with an aspect of this disclosure. In the process 2600, apatron is detected in an area (2605) by, for example, the facialrecognition system, motion sensors, heat sensors, and/or any other typeof sensor that may detect the presence of one or more patrons in theaudience area.

An image is captured of the area (2610), for example, in response to thedetection of one or more patrons in the designated area by image capturedevice 2515. Alternatively, the image capture device may periodicallycapture an image at pre-defined intervals of time, or a video feed ofthe designated area may be continuously captured and one or more imagesmay then be extracted from the video feed when one or more patrons aredetected in the designated area.

The captured image is transmitted to a central control device or module(e.g. 2104, FIG. 21) (2615), optionally with other information about theimage. Such other image information may include, for example, camerasettings, depth information, a time stamp, lighting information, alocation, and/or other like information related to the image. The imageinformation may be transmitted separately, or it may be transmitted inor with the captured image. Optionally, a video feed may be provided tothe central control device or module. The exhibit control device ormodule may optionally monitor a video feed and only send an image thatincludes patrons taken from a feed when a patron is detected in thearea. The exhibit control device or module may optionally perform imageprocessing to improve image quality prior to transmitting the image,and/or it may optionally isolate facial images from the captured imageand send only portions of the image that include facial images to thecentral control device or module. Process 2600 may then end.

Although an exemplary process performed by the exhibit control device ormodule to obtain location information for registered users in accordancewith aspects of this disclosure is discussed above with respect to FIG.26, other processes performed by the exhibit control device or module toprovide the media content that add to, combine, rearrange, or remove anyof the described steps are possible and are considered within the scopeof this disclosure.

In accordance with some aspects of this disclosure, the system may alsostore records of consumable products that a registered user purchasesand/or is awarded. Such records could be entered into a database via oneof the disclosed computer systems that recognizes a transaction andrecords attributes of that transaction in a database. For purposes ofthis discussion, consumable products are any item that may be awarded toor purchased by a registered user in the venue. Examples include, butare not limited to, toys, gift certificates, souvenirs, or any otheritem that a user may obtain in the venue. The user may then go to aredemption center at a later time and obtain the consumable productsthat the user had purchased/received earlier. Preferably, the redemptioncenter would be near the exit of the venue to allow the user to collectthe obtained consumable products just prior to leaving the venue.

FIG. 27 illustrates a flow diagram of a process performed by an exhibitcontrol module or device (e.g. 2108, FIG. 21) to send information abouta consumable product obtained by a registered user at the exhibit. Theprocess 2700 begins by detecting the user is entitled to a consumableproduct (2705). This detection may be a purchase of a consumable productor an awarding of the consumable product as a prize in a game and/orcontest. For example, an image could be taken of the user, and a facialfingerprint could be entered into an image database to retrieve anidentifier of the user, which is then cross-referenced against adatabase that contains relationships between entity identifiers andproduct entitlements. When such an entitlement is returned by thedatabase, this could trigger recognition that the user in the image isentitled to a consumable product identified in the database.

The process then generates a record for the consumable product thatindicates the consumable product contained and the registered user thatobtained the item (2710). The record of the consumable product is thentransmitted to the central control system (2715) and the process 2700ends.

Although an exemplary process performed by the exhibit control device ormodule to provide records of consumable products obtained by users inaccordance with aspects of this disclosure is discussed above withrespect to FIG. 27, other processes performed by the exhibit controldevice or module to provide the media content that add to, combine,rearrange, or remove any of the described steps are possible and areconsidered within the scope of this disclosure.

Similar to the consumable products discussed above, some exhibits and/orareas may have areas available for photo opportunities and have venueemployees and/or professional photographers at the photo opportunityarea to capture an image of a registered users and/or a group of users.In accordance with some aspects of the invention, the system provides asystem for storing the images captured at various photo opportunityareas and making the images available to the registered user or a userauthorized to act for the register user for retrieval and/or purchase.

FIG. 28 illustrates a flow diagram of another exemplary processperformed by an exhibit control system, device, or module (e.g. 2108,FIG. 21); and/or location control system, device, or module to provideimages of a captured user to the central control system, device, ormodule (2104, FIG. 21) for retrieval by the user in accordance with anaspect of the invention.

The process 2800 begins by capturing an image of a registered user at aphoto opportunity area associated with the exhibit/area (2805). Thisimage can be captured with an image capture device that is separateand/or different from the image capture device(s) used for locationdetermination.

The process then identifies each registered user associated with thecaptured image (2810). The identification may be performed by reading abar and/or bar code on a card carried by the user in accordance withsome aspects. In accordance with some other aspects, the identificationmay be performed using the facial recognition system, device, or module(e.g. 2102, FIG. 21). The image and identifiers (FIG. 33) associatedwith each identified registered user in the image are transmitted to thecentral control system (2815). In accordance with some embodiments, theidentifiers of the identified registered users may be embedded in themetadata of the image, or may be added to the metadata of the image bythe system. In accordance with some other embodiments, an image recordthat includes the image and the identifiers may be generated andtransmitted. This metadata may be saved onto a database to allow foreasy-searching of the image. The process 2800 may then end.

A process performed by an exhibit control system, device or module toprovide images of registered users captured in an photo opportunity areain accordance with an aspect of the invention is described above withreference to FIG. 28, other processes performed by the exhibit controlsystem, device, or module to provide the media content that add to,combine, rearrange, or remove any of the described steps are possibleand are considered within the scope of this disclosure.

FIG. 29 illustrates a flow diagram of an exemplary process performed bythe central control device or module (e.g. 2104, FIG. 21) to obtain andstore location information for registered users; and/or obtain and storecrowd information for exhibits and/or locations, based upon capturedimages from various locations throughout a venue. A process 2900 may beperformed for some or all of the images received. Thus, the process 2900may be performed once to determine the current locations of registeredusers, or, alternatively, the process 2900 may be periodically performedon images from all of the locations to constantly update the locationinformation for the registered users.

In the process 2900, captured images of various locations in the venueare received from an exhibit control device or module (e.g., 2108, FIG.21) (2905). As discussed above, additional image information mayoptionally be received with the image. The images are then provided to afacial recognition device or module (e.g. 2102, FIG. 21) for imageanalysis. The central control device or module may perform some imageprocessing prior to providing the images to the facial recognitiondevice or module. The analysis may include, for example, isolatingfacial images in each of the images, modifying the images to improveimage quality, and/or analyzing the images to determine or obtainadditional or other image information. In some embodiments, such otherimage information may be provided by the captured image to the facialrecognition system. The received images from the various locations areprovided to the facial recognition module or system. As will explainedin more detail below, the facial recognition system identifies users ineach image and associates a location of the image with each identifieduser to provide location information for each of the identifiedregistered users. Furthermore, the location information may include atime stamp indicated the time at which the image of a registered user isdetected at a location.

The process 2900 receives location information and identifiers for theregistered users identified in the captured images (2915). The centralcontrol system (e.g. 2104, FIG. 21) updates the location information inthe record of each identified registered user based on the receivedlocation information (2920).

The process 2900 may also receive crowd information (see FIG. 34) foreach location/exhibit from the facial recognition system (2925) andupdate the crowd information in the location record maintained for eachlocation (2930). Locations may be defined as places within an image, buta table or database may be used to cross-reference locations (e.g. x andy location) within an image against identifiers of a location (e.g. anentity in position 4,5 is in seat 4B but an entity in position 2,3 is inseat 3A) Process 2900 may then end.

Although an exemplary process performed by the central control device ormodule to obtain and maintain location information for registered usersand/or crowd information for locations in accordance with aspects ofthis disclosure is discussed above with respect to FIG. 29, otherprocesses performed by the central control device or module to providethe media content that add to, combine, rearrange, or remove any of thedescribed steps are possible and are considered within the scope of thisdisclosure.

FIG. 30 illustrates a flow diagram of an exemplary process performed bya central control module or system (e.g. 2104, FIG. 21) to maintain alist of consumable products obtained by a registered user. The process3000 begins with the central control module or system receiving aconsumable transaction record from an exhibit control system of module(2108, 2112, FIG. 21) (3005). The transaction record includes anidentifier of the consumable product and an identifier of the registereduser entitled to the product. In some instances, the transaction recordmay include other information such a transaction information may show apurchase prices and/or payment, an expiration date/time, or otherinformation related to the product. Any suitable user interface could beused to save the transaction record, such as a transaction device thatsaves metadata related to a transaction to a database accessible by thesystem when the transaction cocurs.

The consumable products information in the record of the identifiedregistered user is updated to include the consumable product (3010). Ata later time, the central control system may receive a request for thelist of consumable products that have been obtained by the registereduser from a redemption center system (not shown) (3015). The redemptioncenter system may include one or more computing devices for enabling theprocess blocks described herein with respect FIG. 30 or otherwise. Insome embodiments, a user having the authorization to obtain products forone or more other users (such as the children of the user or other groupmembers) may obtain the list of consumable products available to eachuser for which the registered user has authorization to obtain theirproducts. The central control system transmits the list of consumableproducts from the user record of the registered user to the redemptioncenter system in response to the request (3020). The redemption centersystem then provides the listed consumable products to the user. Theredemption center system then adds each consumable product received bythe registered user to a redeemed items list and transmits the list tothe central control system (3025). The central control system thenupdates the consumable products list in the user record of theregistered user to reflect the products redeemed (3030) and the process3000 may end.

Although an exemplary process performed by the central control device ormodule to maintain consumable product information for registered usersin accordance with aspects of this disclosure is discussed above withrespect to FIG. 30, other processes performed by the central controldevice or module to provide the media content that add to, combine,rearrange, or remove any of the described steps are possible and areconsidered within the scope of this disclosure.

In accordance with some aspects of this invention, the central controlsystem, device, or module stores images from photo opportunity areas andassociates the images with registered users. The central control system,device, or module then may provide the images to a redemption device forselection, deletion, and/or purchase of the images by the registereduser, for example by configuring a database to return an image as afunction of an identifier of a person (e.g. a bar code, a person'sname).

FIG. 31 illustrates another exemplary process performed by an inventivecentral control system, device, or module (e.g. 2104, FIG. 21) forstoring the images, associating with registered users, and providing theimages to a redemption device in accordance with an aspect of thisdisclosure. In process 3100, an image and identifiers associated withregistered users in the image are received from an exhibit/locationsystem, module, or device (3105). The image is then stored at a memorylocation accessible to a processor (3107). In accordance with someembodiments, the image is stored in an image database, data structureusing structured or unstructured objects or library maintained bycentral control system, device, or module as described below withrespect to FIG. 33. The user record of each registered user identifiedin the image is then updated to indicate storage information for theimage to associate the image with the registered user (3110).

At a later time, the central control system may receive a request forthe list of images that have been associated with a registered user froma redemption device (3115). In some embodiments, a user having theauthorization to obtain/view images associated with one or more otherusers (such as the children of the user or other group members) mayobtain the list of images associated each user for which the registereduser has authorization to view/obtain their images. The central controlsystem transmits the list of images from the user record(s) of theregistered user(s) to the redemption device in response to the request(3120). The redemption device then provides the listed images to therequesting user. This may include the redemption device downloading theimages from the image database and/or requesting the images from thecentral control system. The redemption device then adds each imagereceived, purchased, or uploaded by the registered user to a saved imagelist and transmits the saved list to the central control system; and/oradds each deleted/discarded image to a discarded image list. The savedimage list and/or discarded image list are received by the centralcontrol system (3125). The central control system may then update theassociated images in the user record of the registered user and/orimages stored in memory to reflect each of the received lists (3130) andthe process 3100 may end.

Although an exemplary process performed by the central control device ormodule to store and provide images from photo opportunity areas forregistered users in accordance with aspects of this disclosure isdiscussed above with respect to FIG. 31, other processes performed bythe central control device or module to provide the media content thatadd to, combine, rearrange, or remove any of the described steps arepossible and are considered within the scope of this disclosure.

A registered user may have certain restrictions regarding the exhibitsor locations that the user may visit. For example, some parents may notwant their children to view and/or ride certain exhibits as the materialin the exhibit or ride may not be age appropriate for their children.Also, the venue may want to assure children exit the venue with theappropriate guardian such as a teacher or parent or that the parent orother adult does not leave a minor unattended in all or a portion of thevenue. Thus, systems and processes in accordance with some of theseaspects of the invention may want to monitor the location of aregistered users and determine whether any restrictions and/or accessprivileges are met when a registered user is determined to be at acertain exhibit and/or other location. FIG. 32 illustrates of a processperformed by the central control system to monitor an area in accordancewith an aspect of this disclosure to determine whether a register useris permitted in the area based on restrictions and/or access privileges.

Process 3200, shown in FIG. 32, receives an image captured from anexhibit control module or system (e.g. 2108, 2112, FIG. 21) associatedwith a particular exhibit or location (3205). The image is provided bythe central control system or module (e.g. 2104, FIG. 21) to a facialrecognition system or module (e.g. 2102, FIG. 21) (3210). The centralcontrol system receives location information for each user identified inthe image (3215). The central control system or module then determineswhether the each of the identified users has any restrictions and/oraccess privileges for the particular exhibit and/or location (3220). Forexample, a parent may be restricted from leaving without a minor and/ora minor may be restricted from leaving without an authorized adult. In asecond example, an exhibit may be popular and users are restricted toaccess the exhibit at a certain time. In some cases, the user may havean access privileges, such as a right to enter a premium exhibit and/orride. If there are restrictions and/or access privileges for identifiedregistered user for the particular exhibit location, a check is made todetermine the restrictions/access privileges are met (3225). As a secondexample, the central control system may determine if an authorized adultis with an identified minor and/or if a parent is with their minor at anexit or at an exhibit with material that may not be age appropriate forthe minor. If the restrictions/access privileges are met, process 3200may end. If the restrictions/access privileges are not met, anotification identifying the restriction or privilege is generated(3030). In some aspects, this may cause a warning such as an alarm,buzzer, or some other device to be activated to notify venue personnelof the restriction/access. In some aspects, the notification is sent toa device of the user and/or a designated party in the group and/orsubgroup of the registered user (3235). In some other aspects, thenotification is transmitted to one or more devices of venue personnel atthe exhibit/location to allow the personnel to handle the situation.Process 3200 may then end.

Although an exemplary process performed by the central control device ormodule to monitor restrictions and/or privileges at an exhibit/locationin accordance with aspects of this disclosure is discussed above withrespect to FIG. 32, other processes performed by the central controldevice or module to provide the media content that add to, combine,rearrange, or remove any of the described steps are possible and areconsidered within the scope of this disclosure.

In accordance with some aspects of this disclosure, the central controlsystem may provide information about the locations of registered usersin a group to other registered users in the group. Furthermore, users ingroups may be further divided into subgroups with only certain groupmember having access privileges for location information for the groupsand/or particular subgroups. The location information for members in agroup/subgroup may be conveyed to an application being executed on apersonal device of a registered user or a device at a kiosk in the venuethat has a computer system for providing the location information. FIG.33 illustrates a flow diagram of a process performed by the centralcontrol system or module (e.g. 2104, FIG. 21) to provide locationinformation of a group of registered users in accordance with an aspectof the disclosure.

In process 3300, a request for location information for registered usersassociated with a specific group is received (3305). The user requestingthe information is authenticated (3310). Authentication may be performedusing a password, biometric readings, or in some other manner. If theauthentication fails (3312), process 3300 ends and location informationis not provided. If the requesting user is authenticated, The registeredusers in the requested group/subgroup are then determined (3315), thedetermination may be a list of members maintained for each registeredgroup and may be performed by reading the group and/or subgroup field inthe registered user record of each user in the group maintained by thecentral control system or module. A determination is made whether therequesting user is authorized to obtain the location information (3317).This may be performed by reading the access permissions stored in theregistered user record of the requesting user. In other embodiments, thepermission may be stored in a group data record maintained by thecentral control system, device, or module for the requested group and/orsubgroup. If the requesting user does not have authorization to receivethe information, the process 3300 ends. If the requesting user isauthorized, the central control system, device or module retrieves thelocation information for each registered user in the group and/orsubgroup from the user records maintained by the central control system,device, or module (3320). In some embodiments, the location informationfor each registered user only includes information about the lastdetermined location of the user. In accordance with some otherembodiments, the location information may include information about 2 ormore last locations where the registered user was detected. In stillsome other embodiments, the entire history of locations of eachregistered user in the group is provided. In some further aspects,directions to get to the current location of a particular user or groupof users may be provided. The location information for each of theregistered users in the group is provided to the requesting device(3325) and process 3300 may end.

Although an exemplary process performed by the central control device orprovide location information for a user or group of users in accordancewith aspects of this disclosure is discussed above with respect to FIG.33, other processes performed by the central control device or module toprovide the media content that add to, combine, rearrange, or remove anyof the described steps are possible and are considered within the scopeof this disclosure.

FIG. 34 illustrates an example of a registered user record maintained bythe central control device or module in accordance with an aspect ofthis disclosure. The register record 3400 advantageously includes anidentifier 3405, such as a name or member number for the registereduser. The record 3400 also includes a facial image 3410 of theregistered user, either provided to the central control device ormodule, or captured from the registered user during a registrationprocess. The record 3400 also includes a group field 3425 thatidentifies a group that is associated with the registered user. Inaddition, one or more subgroup fields 3430 may be provided.Alternatively, the subgroup field may be a pointer to or a linked listthat includes all of the subgroups to which the user belongs.

Record 3400 may also include access permission field 3435 andrestrictions field 3440. An access permission field or data structure3435 stores codes indicating all of the access privileges of the user.The access privileges may indicate the exhibits to which the user hasaccess, the rights to access location information of other users orgroups, and/or quality of service indicators such as Very ImportantPerson (VIP) privileges that the user may have in the venue.Restrictions field or data structure 3440 is a list, pointer to a list,or other data structures that includes all of the restrictions thatapply to a user. The restrictions may include, but are not limited to,restrictions on exiting the venue, restrictions on access or access timeto an exhibit, prohibited areas of the venue and the like.

Other fields for particular information about the registered user may besaved in a database. For example, the fields in the record 3400 mayadvantageously include a field for the user's age 3415, a link to a listof user locations 3465, a link to a list of consumable products 3460,list of images associated with the user 3455, list of desired exhibitsto visit 3450 and any special needs of the registered user 3445. Thelist of special needs may include a list of requirements such aslanguage, subtitles, special access and the like that user may requireduring a visit to exhibits in the venue. In some embodiments, the fieldscould be inputted by an admin user, while in other embodiments the usercould input one or more fields in a user interface kiosk at the venue.

Although an example of a registered user data record is described abovewith reference to FIG. 34, other suitable data records that add, remove,and/or combine fields could be utilized.

In accordance with some aspects of the disclosure the central controlsystem, device, or module may also or alternatively provide crowdinformation for particular exhibits and/or other areas of a venue. Forexample, the system may indicate to a registered user when a desiredexhibit is either crowded with long wait time or not crowded with shortwait times. In some embodiments, the system may even suggest an orderingof the exhibits and/or areas to visit to ensure that a registered usercan visit all of the exhibits and/or areas desired by the user in anamount of time allocated by the user. To do so, the central controlsystem (2104, FIG. 21) may maintain a location database that storescrowd information for each location and/or exhibit in the venue. Alocation record in accordance with an aspect of the disclosure is shownin FIG. 35.

Location record 3500 includes a location identifier field 3505, crowdinformation field 3510, and interested user field 3515. The locationidentifier field 3505 is a field that stores a name or some other alphanumeric identifier of the location/exhibit. The crowd information field3510 is a lost or pointer to a list that stores, among other pieces ofinformation, the size of the crowd at certain times as determined fromthe captured images of the location/exhibit. The interested user field3515 is a list, pointer to list or some other data structures thatstores identifiers of registered users interested in visiting theparticular location/exhibit.

Although a location record maintained for a location/exhibit by thecentral control system in accordance with an aspect of the disclosure isdiscussed above with reference to FIG. 35, location records that add,remove, or combine fields are considered part of this disclosure.

FIG. 36 illustrates a flow diagram of an exemplary registration processperformed by a central control system or the central control system,device, or module (e.g. 2104, FIG. 21) in accordance with an aspect ofthis disclosure. In the registration process 3600, a facial image of theuser that is registering with the system is received (3605). Forexample, the user may provide a facial image stored on the user devicethat the user is using to register. In that situation, the process 3600may issue a command (for example, by a wireless communication) thatdirects the user device to capture the facial image using an imagecapture device associated with the user device, and to provide the imageto process 3600. The process 3600 may also receive user information forthe user (3610). The personal information may include, but is notlimited to, age, address, e-mail address, home telephone number,cellular telephone number, language, disabilities, and another specialneeds. In accordance with some aspects, the registering user may inputthe information to a personal device that provides the information tothe process 3600. The process 3600 may also receive group informationand subgroup information (3615) where the subgroup is a set of one ormore users from the group.

The process 3600 may also receive any authorizations for the user(3620). Authorizations may include access to location information forusers in a particular group and/or subgroup as well as authorization tocollect and/or receive consumable items for one or more other users andaccess to images of one or more other users (3625). The process 3600 mayalso receive restrictions for the users. As discussed above,restrictions may include restrictions prohibiting attending particularexhibits and/or areas of the venue as well as restrictions for exitingthe venue. The process 3600 may also receive a list of one or moreexhibits of interest of the user (3630) for use in providing guidance tothe user in attending the exhibits of interest. Further any otherinformation that may include at least a portion of the informationdiscussed above with reference to FIG. 14 may also be received byprocess 3600. However, the information may also include any informationthat may be desired.

A registered user record that includes the received user memberinformation discussed above and the captured facial image is generated(3635) and stored in a user database (3640). The captured facial imageand an identifier of the registered user is provided to the facialrecognition device or model (3645), and the process 3600 may then end.

An exemplary process for registering a user in accordance withembodiments of the disclosure is described above with respect to FIG.36. Other registration processes that add, combine, rearrange, and/orremove one or more steps described above are possible in accordance withother embodiments.

In accordance with some aspects of the disclosure, the systems andprocess may provide information relating to exhibits to registered usersto guide the users through the venue to maximize the users enjoyment byensure that all of the exhibits desired by the user may be visited. Thismay include providing a schedule or map to the user to guide the user tothe venue and/or updating the schedule and/or map based on observedcrowd conditions throughout the venue. In accordance with someembodiments, the system may only provide information regarding crowdinformation regarding particular exhibits to aid the user in determiningwhich exhibits to visit next.

FIG. 37 shows an exemplary process performed by a central controlsystem, device, or module (e.g. 2104, FIG. 21) for informing registeredusers about crowd conditions at an exhibit and/or location in a venue. Aprocess 3700 retrieves crowd information for a particular exhibit and/orlocation from a location record for the particular exhibit and/orlocation (3705). The crowd information may be a crowd size determinedfrom a captured image of an area near and/or in the exhibit and/or thelocation that was stored in the location record, as described above withrespect to FIG. 34. The crowd information is compared to pre-determinedminimum and maximum crowd thresholds (3710). The threshold valuesindicate a crowd size.

If the crowd size is less than a minimum threshold (3712), the process1300 determines a set of the registered users interested in the exhibitand/or location and notifies the each of the registered users in the set(3715). In accordance in some embodiments, a list of interested users ismaintained by the system and a predetermined number from the top of thelist are notified. The registered users on the list may be removed afterit is determined that the user has visited the location, for example, byusing the processes described above. In some other embodiments, all ofthe users on list may be notified and a user is removed from list afterit is determined that the user has visited the location, for example, byusing the processes described above. If the crowd size is greater thanor equal to a maximum threshold (3737), the interested users arenotified that the exhibit and/or location is at or beyond capacity. Inaccordance with various aspects of this disclosure, the notificationsmay be sent to a hand held device via a message such as, for example, anSMS message or an e-mail message. Process 3700 then ends.

An exemplary process for notifying registered users of crowd conditionsat an exhibit and/or location in accordance with embodiments of thedisclosure is described above with respect to FIG. 37. Othernotification processes that add, combine, rearrange, and/or remove oneor more steps described above are possible in accordance with otherembodiments.

FIG. 38 illustrates an exemplary process performed by a facialrecognition device or module (e.g. 2102, FIG. 21) in response toreceiving a facial image and identifier of a registered user inaccordance with embodiments of this disclosure. In the process 3800, thefacial recognition device or module receives a facial image andidentifier of a registered user from a central control system or centralcontrol device or module (3805). A facial recognition record isgenerated and stored (3810). The generation of the facial recognitionrecord may include, for example, performing analysis of the image togenerate facial parameters that may be used for image comparisons duringthe facial recognition process, as discussed further below.

An exemplary process for obtaining facial images of registered users ina facial recognition system accordance with embodiments of thedisclosure is described above with respect to FIG. 38. Other processesfor obtaining facial images that add, combine, rearrange, and/or omitone or more steps described above are possible in accordance with otherembodiments.

FIG. 39 is a conceptual data structure for a facial recognition recordin accordance with an aspect of the disclosure. A facial recognitionrecord 3900 includes an identifier of the registered user 3905, thereceived facial image 3910, and the facial parameters for facialrecognition comparisons 3915. The identifier may be, for example, a nameand/or nickname of the registered user, or the identifier may be anumber or alphanumeric string that associates the image to a specificregistered user record stored by the central control device or module.

Although an exemplary facial recognition record in accordance withembodiments of the disclosure is described above with reference to FIG.39, other facial recognition records that add, combine, rearrange,and/or omit information are possible in accordance with otherembodiments.

The software and/or hardware modules that perform a facial recognitionprocess in accordance with embodiments of the disclosure are shown inFIG. 40. The facial recognition system 4000 includes a receiving module4005, a facial image identifier module 4010, a facial image analysismodule 4015, a facial recognition module 4025, and a crowd informationmodule 4030.

The receiving module 4005 receives a captured image and processes thecaptured image to conform the image to the parameters needed to performthe various subsequent processes for facial recognition analysis. Inaccordance with some aspects, the image processing may include, forexample, focus adjustments, color adjustments, edge defining, and otherimage adjustments needed to conform the image to the requirements of thesubsequent modules. In accordance with some aspects, the receivingmodule also receives image information such as, for example, depthinformation, camera information, and lighting information. The receivingmodule 3605 uses the image information in the image processing toconform the image to the required standards.

The processed image is provided to the facial image identifier module4010, which identifies the portions of the image that include a facialimage. The identification may use edge detection and other varioussearch processes to identify those portions of the image that include animage of a face to which facial recognition may be applied. Inaccordance with some aspects, the facial image identifier may alsoperform some image processing to conform the portions including a facialimage to the requirements of an analysis module.

The facial image analysis module 4015 receives the portions of the imagethat include a facial image and performs analysis on each portion of theimage to generate the data needed by the other modules to generate theinformation required. For example, the image analysis module maygenerate pixel color and vector data needed to perform edge detection,color detection, and the like needed to perform the various subsequentprocesses. In accordance with some aspects, the facial image analysismodule 4015 also receives the image information and/or a complete imagefor use in performing the analysis. The information generated by thefacial image analysis module 4015 is provided to the facial recognitionmodule 4025 and crowd information module 4030.

The facial recognition module 4025 receives the information for eachfacial image and compares the information of each facial image to theinformation for the facial images in each facial recognition record todetermine a match and returns the identifier of each record that matchesone of the facial images from the captured image to a predefined degreeof confidence. To facilitate the comparison, the records may includefacial image data that is precomputed to provide quicker comparisons byeliminating the need to analyze each reference image.

In accordance with some aspects, crowd information module 4030, receivesthe image information and determines the number of visitors (whetheridentified as a registered user or not) that are detected in the image.The crowd information module then uses the determined number of visitorsto determine an estimate of the crowd size at the exhibit and/orlocation.

Although the above description describes modules of a facial recognitionsystem in accordance with an exemplary embodiment of the disclosure,other facial recognition modules that that add, combine, rearrange,and/or omit modules are possible in accordance with other embodiments.

FIG. 41 illustrates a flow diagram of a process performed by the facialrecognition system 4000 to perform facial recognition in a capturedimage of an area in accordance with an aspect of the disclosure. In theprocess 4100, an image of a known location is received (4105). Asdiscussed above, the received image may be processed to conform theimage to the requirements of the process 4100.

Portions of the received (and optionally processed) image that include afacial image are identified (4110). As discussed above, each portion maybe further processed to conform the facial image to the requirements ofthe facial recognition process. A facial recognition comparison to thefacial images stored in the facial recognition record is performed toidentify the records that match the facial images (4115).

The location shown in the image is associated with each identifiedregistered user (4125). The user and location information is thenprovided to the central control system (4130). Crowd information foreach exhibit and/or location associated with each image is determined(4135) and the crowd information for each location is provided to thecentral control system, device, or module. Process 4100 may then end.

An exemplary process for determining user location information and crowdinformation for exhibits and/or location in a facial recognition systemaccordance with embodiments of the disclosure is described above withrespect to FIG. 41. Other processes that add, combine, rearrange, and/oromit one or more steps described above are possible in accordance withother embodiments

Many other variations of the above could be derived using the disclosedcomputer infrastructure and functionality to dynamically deliver mediacontent. Note that references throughout this specification to “oneaspect” or “an aspect” mean that a particular feature, structure orcharacteristic described in connection with the aspect is included in atleast one aspect of the present invention. Therefore, it is emphasizedand should be appreciated that two or more references to “an aspect” or“one aspect” or “an alternative aspect” in various portions of thisspecification are not necessarily all referring to the same aspect.Furthermore, the particular features, structures or characteristicsbeing referred to may be combined as suitable in one or more aspects ofthe present disclosure, as will be recognized by those of ordinary skillin the art.

Below are additional points that could be embodied using the inventivesystems and methods disclosed.

A first point being a system for providing media content, comprising: afirst image capture device that captures a first set of captured images,wherein the first set of captured images comprise a first set of imagefingerprints; a media content playback device; an image database thatcorrelates image fingerprints with identifiers; a relationship databasethat correlates identifiers with one another; an attribute database thatcorrelates identifiers with attributes; a media database that correlatesattributes with media content; and a media content computer systemprogrammed to: generate a first set of identifiers as a function of thefirst set of image fingerprints using the image database, generate asecond set of identifiers as a function of the first set of identifiersusing the relationship database, generate a derived set of attributes asa function of the second set of identifiers using the attributedatabase, generate a set of media content as a function of the derivedset of attributes using the media database; and play at least a portionof the set of media content on the media content playback device.

A second point being a system of the first point, wherein the derivedset of attributes are generated as a function of both the first set ofidentifiers and the second set of identifiers.

A third point being a system of the first point, wherein the first setof identifiers comprise a license plate number and wherein the secondset of identifiers comprise a facial fingerprint.

A fourth point being a system of the third point, wherein the second setof identifiers comprises both a person and a position within a carassociated with the license plate number.

A fifth point being a system of the first point, further comprising asecond image capture device that captures a second set of capturedimages, wherein the second set of captured images comprise a second setof image fingerprints, and wherein the media content computer system isfurther programmed to: generate a correlation set of identifiers as afunction of the second set of image fingerprints using the imagedatabase, generate a set of relationships using the correlation set ofidentifiers, and saving the set of relationships in the relationshipdatabase.

A sixth point being a system of the fifth point, wherein the second setof image fingerprints comprise both a license plate number and a facialfingerprint.

A seventh point being a system of the first point, further comprising: atransactional capture device that captures a set of transactionattributes and a captured transaction identifier, wherein the mediacontent computer system is further programmed to: recognize atransaction in the first set of captured images; generate a generatedtransaction identifier as a function of the recognized transaction;transmit the generated transaction identifier to the transactionalcapture device; receive the set of transaction attributes from thetransactional capture device; and save the set of transaction attributesto the attribute database.

An eighth point being a system of the seventh point, wherein thegenerated transaction identifier and the captured transaction identifierboth comprise a timestamp value, and wherein the transactional capturedevice matches the generated transaction identifier with the capturedtransaction identifier before transmitting the set of transactionattributes to the media content computer system.

A ninth point being a system of the seventh point, wherein the mediacontent computer system recognizes a transaction in the first set ofcaptured images by identifying a movement among a plurality of the firstset of captured images.

A tenth point being a system of the seventh point, wherein the derivedset of attributes comprises at least a portion of the set of transactionattributes.

An eleventh point being a system of the first point, wherein the mediacontent computer system is further programmed to transmit at least aportion of the derived set of attributes to a service computer system.

A twelfth point being a method for generating targeted media content,comprising: capturing a first set of images using a first image capturedevice; identifying a first set of captured image fingerprints withinthe first set of images; deriving a first set of identifiers as afunction of the identified first set of captured image fingerprintsusing an image database; deriving a second set of identifiers as afunction of the first set of identifiers using a relationship database;deriving a first set of attributes as a function of the second set ofidentifiers using an attribute database; deriving a set of media contentas a function of the first set of attributes using a media database; andplaying a portion of the set of media content on a media contentplayback device.

A thirteenth point being a method of the twelfth point, whereinidentifying the first set of captured image fingerprints comprisesidentifying a license plate number within the first set of images,wherein deriving the first set of identifiers comprises deriving anidentifier of a car as a function of the license plate number using theimage database, and wherein deriving the second set of identifierscomprises deriving a set of occupants of the car from the identifier ofthe car using the relationship database.

A fourteenth point being a method of the thirteenth point, furthercomprising: capturing a second set of images using a second imagecapture device; identifying a second set of captured image fingerprintswithin the second set of images, wherein identifying the second set ofcaptured image fingerprints comprises identifying the license platenumber within the second set of images and identifying a set of facialfingerprints within the second set of images; deriving a set of occupantidentifiers as a function of the identified set of facial fingerprintsusing the image database; deriving the identifier of the car as afunction of the license plate number using the image database; andadding a relationship between the identifier of the car and theidentified set of facial fingerprints to the relationship database.

A fifteenth point being a method of the thirteenth point, furthercomprising: identifying a transaction in the first set of images;retrieving a set of transaction attributes from a transactional capturedevice as a function of the identified transaction; adding the set oftransaction attributes to the attribute database as a function of thesecond set of identifiers.

A sixteenth point being a method of the thirteenth point, furthercomprising: deriving an average speed of the car from at least two ofthe first set of images, wherein the media content playback devicecomprises a roadside billboard; and deriving a time as a function of theaverage speed and a distance to the roadside billboard, wherein playingthe portion of the set of media content comprises delaying a time toplay the portion of the set of media content as a function of thederived time.

A seventeenth point being a method of the twelfth point, whereinderiving a first set of attributes comprises deriving the first set ofattributes as a function of both a portion of the first set ofidentifiers and the second set of identifiers using the attributedatabase.

An eighteenth point being a method of the twelfth point, furthercomprising: identifying an interaction with the media content playbackdevice within the first set of images; and playing a different portionof the set of media content on the media content playback device as afunction of the interaction.

A nineteenth point being a method of the eighteenth point, wherein theinteraction comprises an action of looking away from the media contentplayback device.

A twentieth point being a method of the nineteenth point, furthercomprising updating a set of attributes associated with at least one ofthe first set of identifiers and the second set of identifiers as afunction of the interaction.

What is claimed is:
 1. A system for providing media content, comprising:a first image capture device that captures a first set of capturedimages, wherein the first set of captured images comprise a first set ofimage fingerprints; a media content playback device; an image databasethat correlates image fingerprints with identifiers; a relationshipdatabase that correlates identifiers with one another; an attributedatabase that correlates identifiers with attributes; a media databasethat correlates attributes with media content; and a media contentcomputer system programmed to: generate a first set of identifiers as afunction of the first set of image fingerprints using the imagedatabase, generate a second set of identifiers as a function of thefirst set of identifiers using the relationship database, generate aderived set of attributes as a function of the second set of identifiersusing the attribute database, generate a set of media content as afunction of the derived set of attributes using the media database; andplay at least a portion of the set of media content on the media contentplayback device.
 2. The system of claim 1, wherein the derived set ofattributes are generated as a function of both the first set ofidentifiers and the second set of identifiers.
 3. The system of claim 1,wherein the first set of identifiers comprise a license plate number andwherein the second set of identifiers comprise a facial fingerprint. 4.The system of claim 3, wherein the second set of identifiers comprisesboth a person and a position within a car associated with the licenseplate number.
 5. The system of claim 1, further comprising a secondimage capture device that captures a second set of captured images,wherein the second set of captured images comprise a second set of imagefingerprints, and wherein the media content computer system is furtherprogrammed to: generate a correlation set of identifiers as a functionof the second set of image fingerprints using the image database,generate a set of relationships using the correlation set ofidentifiers, and saving the set of relationships in the relationshipdatabase.
 6. The system of claim 5, wherein the second set of imagefingerprints comprise both a license plate number and a facialfingerprint.
 7. The system of claim 1, further comprising: atransactional capture device that captures a set of transactionattributes and a captured transaction identifier, wherein the mediacontent computer system is further programmed to: recognize atransaction in the first set of captured images; generate a generatedtransaction identifier as a function of the recognized transaction;transmit the generated transaction identifier to the transactionalcapture device; receive the set of transaction attributes from thetransactional capture device; and save the set of transaction attributesto the attribute database.
 8. The system of claim 7, wherein thegenerated transaction identifier and the captured transaction identifierboth comprise a timestamp value, and wherein the transactional capturedevice matches the generated transaction identifier with the capturedtransaction identifier before transmitting the set of transactionattributes to the media content computer system.
 9. The system of claim7, wherein the media content computer system recognizes a transaction inthe first set of captured images by identifying a movement among aplurality of the first set of captured images.
 10. The system of claim7, wherein the derived set of attributes comprises at least a portion ofthe set of transaction attributes.
 11. The system of claim 1, whereinthe media content computer system is further programmed to transmit atleast a portion of the derived set of attributes to a service computersystem.
 12. A method for generating targeted media content, comprising:capturing a first set of images using a first image capture device;identifying a first set of captured image fingerprints within the firstset of images; deriving a first set of identifiers as a function of theidentified first set of captured image fingerprints using an imagedatabase; deriving a second set of identifiers as a function of thefirst set of identifiers using a relationship database; deriving a firstset of attributes as a function of the second set of identifiers usingan attribute database; deriving a set of media content as a function ofthe first set of attributes using a media database; and playing aportion of the set of media content on a media content playback device.13. The method of claim 12, wherein identifying the first set ofcaptured image fingerprints comprises identifying a license plate numberwithin the first set of images, wherein deriving the first set ofidentifiers comprises deriving an identifier of a car as a function ofthe license plate number using the image database, and wherein derivingthe second set of identifiers comprises deriving a set of occupants ofthe car from the identifier of the car using the relationship database.14. The method of claim 13, further comprising: capturing a second setof images using a second image capture device; identifying a second setof captured image fingerprints within the second set of images, whereinidentifying the second set of captured image fingerprints comprisesidentifying the license plate number within the second set of images andidentifying a set of facial fingerprints within the second set ofimages; deriving a set of occupant identifiers as a function of theidentified set of facial fingerprints using the image database; derivingthe identifier of the car as a function of the license plate numberusing the image database; and adding a relationship between theidentifier of the car and the identified set of facial fingerprints tothe relationship database.
 15. The method of claim 13, furthercomprising: identifying a transaction in the first set of images;retrieving a set of transaction attributes from a transactional capturedevice as a function of the identified transaction; adding the set oftransaction attributes to the attribute database as a function of thesecond set of identifiers.
 16. The method of claim 13, furthercomprising: deriving an average speed of the car from at least two ofthe first set of images, wherein the media content playback devicecomprises a roadside billboard; and deriving a time as a function of theaverage speed and a distance to the roadside billboard, wherein playingthe portion of the set of media content comprises delaying a time toplay the portion of the set of media content as a function of thederived time.
 17. The method of claim 12, wherein deriving a first setof attributes comprises deriving the first set of attributes as afunction of both a portion of the first set of identifiers and thesecond set of identifiers using the attribute database.
 18. The methodof claim 12, further comprising: identifying an interaction with themedia content playback device within the first set of images; andplaying a different portion of the set of media content on the mediacontent playback device as a function of the interaction.
 19. The methodof claim 18, wherein the interaction comprises an action of looking awayfrom the media content playback device.
 20. The method of claim 19,further comprising updating a set of attributes associated with at leastone of the first set of identifiers and the second set of identifiers asa function of the interaction.